What Happens When Nonprofits Use Quality Models for Self-Assessment?

What Happens WhenNonprofits Use Quality

Models for Self-Assessment?Rob Paton, Jane Foot, Geoff PayneTwo tools to assist organizational self-assessment have recentlybecome popular in the U.K. nonprofit sector, providing theopportunity for a comparative study of how such tools are usedin practice. Because both spring from the quality movement inmanagement, the study also provided a chance to observe howquality ideas evolve as they are applied in nonprofit contexts.The study found that self-assessment can indeed enable con-structive review processes in a wide range of contexts. However,when the models “work” it is in rather different ways than issuggested by the terms in which they are promoted.

TOOLS to assist in organizational self-assessment have long beena feature of the management literature. Entire books havebeen made up entirely of diagnostic checklists (Rowntree,

1996). In recent years many such tools have been developed for non-profits; examples include a risk-assessment tool (Hassell, 1998), boardreview tools (Harris 1992; National Center for Nonprofit Boards,1999), and Drucker’s general nonprofit management assessment tool(Drucker, 1998; Stern, 1998). Despite this proliferation, little isknown about what happens when a nonprofit tries to make serioususe of a self-assessment model. What does the process characteristi-cally involve? Is this kind of self-scrutiny a way of obtaining the sortof advice offered by a management consultant, but at less cost? Or isit a false comfort? Do the tools deliver the benefits they claim?

These are significant, topical issues in the United Kingdom,where two tools for organizational self-assessment have recentlyattracted widespread interest among nonprofits (Astbury andMayall, 1997; BMI Consulting Group, 1998; Charities Evaluation Ser-vices, 1998; Darvill, 1998; Quality Standards Task Group, 1998;Samuels, 1998). These tools are the Practical Quality AssuranceSystem for Small Organisations (PQASSO) (Farley, 1997) and the

NONPROFIT MANAGEMENT & LEADERSHIP, vol. 11, no. 1, Fall 2000 © Jossey-Bass, a Wiley company 21

Excellence Model (EM) (British Quality Foundation, 1997). Bothhave their origins in the quality movement, which has had mixedreviews from those who write about public and nonprofit manage-ment, organization, and policy. Advocates (Martin, 1993; Kearns, andothers, 1994; Morgan and Murgatroyd, 1994), who are usually writ-ing from a practitioner perspective, claim that it focuses attention onimportant issues, especially insofar as it places the concerns of ser-vice users (as customers) center stage. Sceptics (Kirkpatrick and Mar-tinez-Lucio, 1995; Clarke and Newman, 1997) have seen the qualityconcept as loose, confused, and rhetorical and have viewed theadvent of quality ideas as another expression of the new manageri-alism, best understood in terms of the struggle for legitimacy betweenmanagerial and professional rationalities. Hence research on howthese diagnostic tools are being used is also a chance to track thespread and influence of “quality” ideas within nonprofits.

After describing the two models and reviewing such research ashas been done on self-assessment, the aims and methods of the studyare described, after which the findings are presented and discussed.

The Two Models: Origins and IssuesPQASSO addresses sixteen different quality areas, such as user-centered service, the management committee (board), and the man-agement of resources. For each of these the workbook sets out astandard, defines the terms used in the standard, sets out three lev-els of achievement, and suggests the sort of documentary evidencean organization might provide in order to show that it was consis-tently operating at that level. It is up to the organizations themselvesto decide which areas are important for them and the levels theyneed to achieve. Level 1 is considered adequate for one-year projects,level 2 for most ongoing projects affiliated with federal organizations,whereas level 3 is appropriate for organizations with a professionalmanagement structure. The pack is attractively designed and offersguidance on how it can be used participatively with staff and boardmembers.

The EM was developed by senior quality managers from leadingEuropean companies. It is, in effect, a modified version of the assess-ment framework for the Baldrige Award—the U.S. government’snational quality award for businesses (Ghobadian and Woo, 1994).The EM is structured in terms of nine elements: five “enablers” (howthings are done) and four “results” (actual achievements). These aresubdivided into thirty-two subcriteria, each of which generates a setof scaled questions. The questions—along with user guidance, exam-ples of excellence for the different criteria, graphical display facili-ties, and so forth—are presented on disk and can be used in oneof three ways: (1) by individuals, (2) by teams (with or without facil-itation), and (3) by teams with external validation of the scores. Theresulting scores, in total and in terms of the nine elements, can then

22 PAT O N, FO O T, PAY N E

Research on howthese diagnostictools are beingused is also a

chance to trackthe spread and

influence of“quality” ideas

within nonprofits

be used to track progress over time. Comparisons with other organi-zations in the same industry or size range can also be made throughthe British Quality Foundation’s database of scores from hundreds ofU.K. companies, public bodies, and nonprofits.

Both models embody assumptions about what is required toachieve and maintain high-quality services, but these assumptionsare not the same. Whereas PQASSO has its origins in the structural,somewhat mechanistic, thinking of quality assurance, though con-textualized for small, nonprofit organizations, the EM reflects thebroader, more cultural approach of total quality management andthe experience of large (usually manufacturing) companies.

Most of the literature relevant to these tools concerns theBaldrige model and its derivatives (Ghobadian and Woo, 1994). Evenin the manufacturing contexts where they originated, the validity ofthese models has been controversial. Are they too broad (by includ-ing a range of management processes beyond those associated withquality control) or too narrow (by not adequately addressing suchvital processes as finance, R&D, and marketing)? (For a discussionsee Garvin [1991] or Conti [1997]). And does conforming to themodel actually help performance? Critics point to the business dif-ficulties experienced by some award winners; advocates claim that,overall, success in achieving the award is associated with superiorbusiness performance; others say the evidence is inconclusive(Ghobadian and Woo, 1994). Clearly, establishing the validity of themodels on which self-assessment tools are based is deeply problem-atic, even in contexts where performance may be more easily and sys-tematically measured than in areas of nonprofit activity.

A much more limited strand in this literature concerns the waythe models are used, which turns out to be important. Critics havebeen concerned that the scoring systems associated with quality awardprocesses and comparisons stimulate efforts at looking good orimpressing superiors rather than helping identify weaknesses (Conti,1997). Defenders say the criteria that are “strongly prescriptive onphilosophy [but] open-minded about practices and procedures”(Garvin, 1991, p. 82) are appropriate. And anyway, defenders say, thevast majority of those who use the models do so with a view to self-assessment and performance improvement, not in order to enter theaward process.

Scoring systems aside, reports by independent consultants sug-gest that the self-assessment process can be constructive (for exam-ple, Armistead, 1998) but that it is not always so. Reflecting onextensive experience in the private and public sectors, one consul-tant (Seddon, 1998, p. 32) reports:

There are larger organizations that have taken to the EMlock, stock and barrel; in some it has become mandatory formanagers to conduct a Self-Assessment on an annual basis.In our experience, many become preoccupied with doing

WH E N NO N P R O F I T S US E QU A L I T Y MO D E L S F O R SE L F-AS S E S S M E N T 23

Most of theliterature

relevant to thesetools concernsthe Baldrigemodel and itsderivatives

things right—meeting the expectations of the hierarchy, con-ducting a Self-Assessment, reporting a score and having aplan of action against the model—rather than doing the rightthings. Few find themselves stimulated to question the veryassumptions they have been using to run the organization.

Finally, existing research and reporting on self-assessment,whether in commercial or nonprofit contexts, concerns particulartools—there are no comparative studies—and reports are often pro-vided by “product champions.” The limitations of such reporting areconsiderable (Brunsson and Olsen, 1993).

Research Questions and MethodsThe intention of the current research was to develop practical theory,that is, propositions that would fit many facts, using concepts thatwould be meaningful for those involved, concerning what happenswhen nonprofits use the two tools described to undertake a processof self-assessment. Comparing two different tools would allow theresearch to focus on the self-assessment process; the fact that bothtools sprang from the quality movement would allow some explo-ration of the way quality ideas were being adopted and adaptedwithin the sector.

The initial aim was descriptive and interpretive: to document howthe tools were being used and to report the sorts of challenges and ben-efits that nonprofit managers and staff associated with these forms ofself-assessment. However, in order to address the limitations of theexisting practitioner literature, the inquiries would have to draw onthe views of multiple informants, not just product champions, and, asfar as possible, track or reconstruct how the self-assessment processunfolded over time. In addition, the study aimed to be analytic: to iden-tify (or develop) concepts that would help illuminate the experiencesand processes reported and place them in theoretical perspective.

The research started with the gathering and studying of the prac-titioner literature (brochures, the trade press, evaluation reports, andso on) concerning the two models and interviewing consultantsand leading practitioners. Thereafter, as the research concerned orga-nizational process, a case study approach was adopted (Yin, 1984).

Case Study Selection and MethodFour organizations were chosen; for each model, one was a single(holistic) case and one a multilevel (embedded) case (Yin, 1984).

Hillend is a drop-in center for marginalized people (addicts,rough sleepers, and so on) that has been running for fifteen years inthe south of England. Its paid staff (four at the time of the study) relyon the assistance of volunteers. It was chosen as a free-standing andrelatively long-term user of PQASSO, having been one of the pilotsites for its development.


Existing researchand reporting onself-assessment,

whether incommercial or

nonprofit contexts,concerns

particular tools—there are nocomparativestudies—and

reports are oftenprovided by

“productchampions”

The Birmingham project is a capacity-building scheme that pro-motes the use of PQASSO in local agencies in England’s second city.The project provides some funding and initial facilitation, as wellas the PQASSO materials, and also supports local networking amongPQASSO users. It has been oversubscribed, attracting involvementfrom a diverse range of agencies and projects, four of which wereselected for interview as part of the case study.

Enterprise and Employment Training Agency (EETA) is agovernment-funded, local agency providing enterprise support anda range of employment-related training in a large town in theMidlands. It employs eighty people and was chosen as a smaller orga-nization that had used the EM, apparently successfully, to helpaddress particular challenges. Again, it had the advantage of being arelatively long-term user of the EM.

Ability is a large national charity working in the field of a par-ticular disability. It provides a wide range of services (residential andcommunity care, schools, employment projects) and operates onabout one hundred sites nationwide. It had the attraction of just start-ing to use the EM as the research began. Hence it offered the possi-bility of a longitudinal, as opposed to a retrospective, study.

Cases in which organizations considered self-assessment butchose not to pursue it or when use was abandoned after a shortperiod were not included. Such instances were encountered followingthe departure of the model’s internal champion. The cases were inves-tigated through visits to obtain relevant documents and to conductinterviews with a cross-section of individuals, using a loosely struc-tured protocol.

Follow-up telephone calls were often used to explore particularpoints or to check on recent developments. In three cases a com-posite account was prepared and sent back to informants forchecking, which often led to further discussion (at Ability the longi-tudinal study continues in the form of occasional visits and inter-views to track what is a fitfully continuing process). Propositionsemerging in one location were tested and elaborated in the light ofexperience in the other sites, and they could also, to a degree, bechecked against the views of consultants, other researchers, andthe professional and research literature from public or commercialcontexts.

Findings: The Practice of Self-AssessmentThe main findings are presented in terms of five propositions. Someof the data that underpin them are provided to illustrate the points.

Proposition 1It is not self-evident how a self-assessment model should be applied to agiven context. The process involves tacit or explicit negotiation overhow the terms of the model should be interpreted and whether par-ticular elements are appropriate.


Cases in whichorganizations

considered self-assessment but

chose not topursue it or when

use wasabandoned aftera short period

were not included

Managers and staff frequently criticized the language ofthe models for being unclear and unsuitable. This was seen as oneof the major challenges concerning use of the model in the pro-posal to adopt the EM made to the Council of Ability. At EETA onemember of staff trained to be an assessor volunteered that shefound the language “quite difficult” and “manufacturing oriented,”which meant they had been “tempted to rewrite all the questions.”She had “put it in layman’s terms for the administrative staff,”though it was “OK for those used to thinking at a strategiclevel.” Similar concerns were expressed regarding PQASSO in theBirmingham project, despite the efforts that had gone into tailoringthe model to the context of small nonprofits (these concerns werealso a major theme in a report on an area trial of PQASSO; seeMcTiernan, 1998).

However, it is unlikely that this issue could be resolved throughclearer expression or more detailed guidance, as participants oftensuggested. Many of the concepts (like “quality” itself) are multifac-eted, and the latitude built into both models is there for good rea-son: in order to accommodate varied circumstances and differentways of doing things and to ensure those undertaking the self-assess-ment instill the terms with their own meanings. Hence some debateover how these general terms will be interpreted and applied in agiven situation is a necessary part of learning to work with themodel.

Proposition 2Perceived benefits from the use of the models seem to arise from the dia-logues to which they give rise and from the new thinking triggered by theterms of the self-assessment process.

Self-assessment seems to have created occasions in whichindividuals expressed preexisting concerns that they had beenunwilling or unable to put across in the usual decision-makingforums. For example, at EETA the self-assessment provided anopportunity for staff to express dissatisfactions, leading to a shared,“independent” and later externally verified confirmation of long-suspected weakness sufficient to convince senior management ofthe need for action. Likewise, staff at Hillend used PQASSO as away of prompting their management committee, which had just“gone on from year to year” with no clear grasp of roles andresponsibilities, to review how they worked. By making neglectedor excluded topics discussable, self-assessment allowed some newideas to emerge or existing ideas to gain wider support. In addition,the conceptual content of the models seems sometimes to havestimulated fresh ways of thinking about the organization andits challenges. This was emphasized at Ability (where the modelwas described as a “thinking tool”), but it showed up in very obvi-ous ways among several PQASSO users who noticed gaps in theirarrangements.


Perceivedbenefits from theuse of the models

seem to arisefrom the

dialogues towhich they give

rise and from thenew thinking

triggered by theterms of the self-

assessmentprocess

Proposition 3The sustained use of both models requires considerable time and effort,and involves the creation of new sorts of discussions whose relationshipto normal decision processes may become problematic.

Narrowly conceived, using the models involves only the timerequired for the self-assessment process and ends with agreement onareas requiring improvement. But interviewees generally took abroader view, including the time and effort spent on designing andimplementing improvements, as part of using the model. In addition,most saw this as an iterative process of diagnosis and improvementrather than a one-time review. For these reasons, what seemed ini-tially to be a fairly limited exercise ended up being a major organi-zational commitment. At Hillend, one area (environmental policy)had yet to be addressed, and at the time of the fieldwork they nolonger expected to complete the process by their target date. Like-wise, although they had decided to review two policy areas at a timeat the six-weekly management committee meetings (thus coveringall areas on an annual cycle), this had yet to start; one respondentwas uneasy about the slow progress. In order to progress as far asthey had, new ways of generating policy and procedure had beenintroduced.

At EETA, the process was driven by a carefully chosen projectteam, made up on a “diagonal slice” basis. Although this was suc-cessful in reducing cynicism, the process experienced a severe lossof momentum when the results were passed to the EETA board, thenpreoccupied with the latest annual work-planning and reportingrequirements coming from Central Government. In the end it tookvarying degrees of effort over three and one-half years to introducethe EM, achieve the first set of verified scores, and obtain board-levelendorsement of the improvement agenda produced through self-assessment.

Again, these time frames do not seem unusual, especially inlarger organizations. A tension between EM-related diagnosis andaction planning and the organization’s established decision processeshas also been noted by Armistead (1998).

Proposition 4The models can be and are used in quite different ways and for differentinternal purposes; within a single organization the manner of and ratio-nale for use may change over time or differ between subunits.

The variability that this gives rise to is striking. In Birmingham,an all-volunteer credit union used PQASSO slowly and deliberatelyover a two-year period to create explicit systems and procedures.The manager and the administrator of an established hostel used itbriskly between them as a diagnostic (“a quick tick box enabled usto see what was needed”) and then set about developing proceduresand practices to fill the gaps. A ten-year-old, ethnic-minority,


women’s educational project, whose founder was the only staff mem-ber, seems to have used it to shift from tacit and personalized waysof working to more explicit and shared arrangements. And a socialcare agency used it for classic QA purposes as the vehicle to obtainregistration with, and hence contracts from, the city social servicesdepartment.

Ability’s expectations around how the EM would be usedchanged even as it was being rolled out, with an initial interest inscoring for external comparability being downplayed and put off toan indeterminate future. Then a few months later, anticipating diffi-culties with his large governing body on which users and carers wereheavily represented, the chief executive seized on the EM as a meansof raising council members’ sights from concerns with particular localservices to key issues of strategy and policy. He made some changesto the representation of the model and used it as the basis for a keysession at a residential meeting where new members were beinginducted and the council’s work for the year was being planned. Thiswas generally agreed to have been very successful (though the staffof Ability who had been trained in the “proper” use of the EM wereshocked at the cavalier way he treated it).

At EETA, a major consideration driving the use of the EM wasthe need to bind together the staff and activities of the three belea-guered organizations out of which the agency had been formed. Andsome of the choices over how the EM might be used were recognizedquite explicitly: “We could have gone for seventy-three smallimprovements or three large ones—we went for the latter” as thechief executive put it.

Notwithstanding this variation, one external purpose was moreor less constant. Indeed, it was highlighted in the promotional mate-rials for both models. Use of the tools was intended to secure theconfidence of external stakeholders. For example, the issue of cred-ibility and how to overcome the “risk-averse” behavior of the urbanregeneration funding agency in relation to small nonprofits was cen-tral to the Birmingham project. At Hillend the decision to tryPQASSO was made when the center’s funders were beginning towrite references to quality assurance into contracts. As regards theEM, it was widely acknowledged that the introduction of a statutoryrequirement for local authorities to demonstrate “best value” hadgiven the use of the EM an enormous boost. It was seen as one wayof helping meet that requirement.

Proposition 5The structure and content of both models provide an integrative map oroverview of management issues that many users value highly; this seemsimportant in understanding the appeal of the models and why generalmanagers, in particular, become committed to them.

The number of spontaneous references to the “broad picture”and “framework” that these models provided was striking, both in


Use of the toolswas intended to

secure theconfidence of

externalstakeholders

interviews in the case organizations and in other discussions.A repeated claim was that they showed how other requirements andimprovement methods (such as contractual monitoring returns,investors in people, quality standards, customer care, and so on) fit-ted together. In EETA, for example, “initiative overload” was a majorissue and initially a barrier to the use of the EM. One of the reasonsthe EM was accepted was precisely because it set the other initiativesin context: “a way of pulling all our quality work together under oneumbrella.” This benefit is referred to in other reports on both the EM(Darvill, 1998; Samuels, 1998) and PQASSO (McTiernan, 1998).

The unprompted frequency of such observations in interviewssuggests that this is a significant benefit. A major challenge facingthose who used these tools was to structure an agenda for improve-ment in a coherent and meaningful way in uncertain and demand-ing circumstances. Examples are diffuse cross-functionalresponsibilities in larger organizations or the extensive roles withmany pressing day-to-day demands of managers in small organiza-tions. Arguably, these managers were using the “maps” provided bythe tools to achieve greater clarity and focus regarding an importantarea of their work, thereby reducing their own role confusion anduncertainty. Though it goes beyond the data and research design ofthis study, it would hardly be surprising if they discharged those rolesmore productively as a result.

DiscussionThe promotional claims for the models imply that by following a rec-ommended procedure and responding to a series of clear and specificquestions, a dispassionate and reliable appraisal will be obtained.Clearly, however, self-assessment is a loose, uncertain, and variableprocess. To account for the diverse experiences and outcomes, oneneeds to consider how use of the models is absorbed into and haseffects on existing patterns of managerial thinking and action in orga-nizations that use them. At one extreme, where the cognitive andbehavioral world is closed and defensive—a Model I theory-in-usein Argyris and Schön’s terms (1978)—managers would either rejectthe questions presented by the model as not meaningful in their sit-uation or they would construe them within existing assumptions, useonly currently available and acceptable information, and discuss theresults in the usual decision-making forums. Such a process mightbe self-congratulatory or just safely banal and unproductive (“coulddo a bit better all round”). Or it might quickly become an occasionfor familiar skirmishes, as preexisting agendas were discretelyexpressed and pursued through the self-assessment process.

At the other extreme, managers within an “open” culture (Model IItheory-in-use) could be expected to engage with the challenge of think-ing in somewhat different and problematic terms about their organi-zation and to suspend judgment about its appropriateness. They would


be more interested in new information about their organization’s func-tioning and performance, and they would be ready to share and discussthis in an appropriate setting outside of the usual meetings cycle. Theywould be willing to use the exercise to surface different perspectivesand concerns and to discuss the information and experience on whichthese rested.

Between these poles are a wide range of possibilities. Even if theculture initially constrains how the model is interpreted and used, itmay still introduce some new constructs, throw new light onneglected information, and precipitate discussions between peopleand on topics that would not otherwise have occurred; new under-standings and action plans may emerge (“sensemaking” is the termused by Weick, 1995).

Thereafter the process may atrophy, or one thing can lead toanother, with further issues being recognized or becoming “discuss-able.” In short, use of the model may stimulate temporarily or moreenduringly some features of a learning organization (Senge, 1990),but whether and to what extent this happens is likely to depend onthe preexisting culture and to be path-dependent on the manner ofits introduction. For example, one might predict that new thinkingand dialogue would be more likely to occur if the model is used, atleast initially, in a context (such as an “away-day,” retreat, or plan-ning conference) that is less overshadowed by the current manage-ment agenda and its accompanying political mind-sets.

Clearly, this is a reconstruction of what may be going on thatneeds to be tested in further research, preferably with different self-assessment tools. However, it fits with other theory and research, forexample, Powell’s proposition (1995) that, for cultural reasons, TQMis not easily imitated, and Harris’s description of her diagnostic pro-cedure as both “a framework for analysis of existing organizationalarrangements” and a “technique for organization development”(1993, p. 276).

In addition, this account of self-assessment processes throwslight on how ideas from the quality movement are being adopted andused in nonprofits. Although a concern for external legitimacy hasbeen significant in the rapid spread of the models as institutional the-orists would predict (see, for example, Di Maggio and Powell, 1983;Lowndes, 1996), it is doubtful whether TQM (total quality manage-ment) and QA (quality assurance) are being spread through theadoption of these forms of self-assessment. The TQM origins of theEM were not widely understood, and indeed, at EETA, the word qual-ity had already been discredited (“In promoting the model to staff,we avoided it like the plague,” according to the model’s internalchampion). As for PQASSO, although it uses the terminology of QA,it is unclear whether, in this attenuated form, it meets the specifica-tion for QA systems given by the International StandardsOrganization, nor is it often being used for conventional QA pur-poses (most users did not have contractual relationships involving


Even if theculture initiallyconstrains how

the model isinterpreted andused, it may stillintroduce somenew constructs,throw new light

on neglectedinformation, and

precipitatediscussions

between peopleand on topics

that would nototherwise haveoccurred; new

understandingsand action plans

may emerge

detailed service specifications to which they had to ensure unvary-ing conformity).

Instead, ideas from QA and TQM are being translated andenacted to make sense and be “workable” in diverse nonprofit cir-cumstances. In the process, the meanings and purposes for which themodels were originally intended to be vehicles are being reconfiguredin line with the evolving preoccupations and purposes of those whouse them. Managers and staff seem to find quality concepts useful,but this is not because they convey a particular way of managing.Rather, these concepts are loose and uncertain. Thus they bringtogether and afford legitimacy to the diverse range of an organiza-tion’s commitments without giving priority to any particular one.Also they allow the recognition of existing good practice and achieve-ments. So although the labels and language of quality are spread-ing, from the evidence of this study, what these terms signify isloose, evolving, and varied. Hence such institutional isomorphism(Di Maggio and Powell, 1983) as is occurring may well be superficial.

ConclusionsThe two quality models discussed were found to be useful vehiclesfor organizational review in diverse nonprofit settings.

Implications for ManagersIf it is thought timely for board members, managers, and staff tostand back and appraise their arrangements, then use of such self-assessment tools may be a relatively economical and reliable alter-native to engaging a consultant; some elements of external facilitationwill usually make that process more productive.

Nevertheless, self-assessment carries no guarantees. Informantsattributed many things to the efficacy of the models, but these toolsdo not themselves “do” anything beyond providing a stimulus andstructure for a certain sort of discussion. That discussion may or maynot be productive, depending on how skillfully it is conducted,whether enough time and effort is afforded to it, and how willingthose involved are to think afresh and engage with the issues thatarise. If the discussion is productive in generating new insights andagreement on priorities for improvement efforts, it may generate aconsiderable change agenda.

A particular value of quality models at the present time is theircredibility in the wider society. The language of quality sustains apublic discourse in terms that are specific, categorical, and definitive.At the same time it enables a private, intraorganizational negotiationthat recognizes the shifting, problematic, and multifaceted nature ofquality in human services. This seems advantageous, allowing man-agers both to address substantive (if often intractable) issues and torepresent their organization externally as successful, client-focused,and abreast of the times.


Ideas from QAand TQM are

being translatedand enacted tomake sense andbe “workable” indiverse nonprofit

circumstances

Implications for Future ResearchTo date, most research on self-assessment has been positivist in orien-tation, based on (or testing) the supposition that organizations bring-ing their arrangements in line with a particular model by a “proper”use of self-assessment (and follow through on it) will achieve improvedperformance. This is also how those who use the models tend to viewthem. However, the propositions emerging from this study do not fiteasily with such assumptions, and it seems that a constructivist per-spective, as represented by institutional theory (Powell and Di Mag-gio, 1991) and by various theorists of organizational process (forexample, Weick, 1995) provides more useful concepts and insights.

From this perspective, the models embody the beliefs and val-ues of loose-knit managerial and policy communities, and thenotions of quality, performance, good management, and so on thatthey are designed to transmit are all socially constructed. Assuch, they are open to continual renegotiation, and have no agreed-upon referents when used in new and different settings. The models“work” to the extent that they facilitate new dialogues around issuesfelt to be important, new ways of making sense of those issues, andnew plans for how they may be addressed. There may not be anyrelation between use of the model and the introduction of particularsorts of changes, let alone improved performance, although thesecould well follow. This perspective prompts different questions forfuture research: Are funders and contractors actually impressed bythe use of quality methods and for how long? Are self-assessment andquality initiatives “decoupled” from the day-to-day work, which con-tinues largely unchanged? Or do they have an impact on task per-formance and client service? Can self-assessment sustain productivedialogue and sense making for long periods, or does it quicklybecome routinized?

ROB PATON designs management development courses at the United King-dom’s premier distance-learning institution, the Open University. He hasbeen researching organization and management issues associated withdifferent sorts of social enterprises for a quarter of a century. He is cur-rently head of the Management Learning R&D Unit at the Open Uni-versity Business School.

JANE FOOT is an independent policy and research consultant in the UnitedKingdom. She works with local government and nonprofit organizationson issues of strategic and corporate policy, service improvement, andcommunity accountability. She has published a number of practicalguides for local government managers on benchmarking and “best value”legislative requirements.

GEOFF PAYNE is a visiting research fellow at the Open University Busi-ness School and project manager at the Charity Finance Directors’


A particularvalue of qualitymodels at thepresent time istheir credibility

in the widersociety

Group, where he is developing a performance resource center for charitieswith funding from the National Lottery Charities Board.

References

Argyris, C., and Schön, D. A. Organizational Learning: A Theory ofAction Perspective. Reading Mass.: Addison-Wesley, 1978.

Armistead, C. “The EFQM and Royal Mail: Changing Perspec-tives and Attitudes.” Paper presented at the “PerformanceMeasurement—Theory and Practice” conference, Centre forBusiness Performance, Judge Institute of Management Studies,Cambridge, England, 1998.

Astbury, R., and Mayall, H. A Map of Quality Standards. London:NCVO Publications, 1997.

BMI Consulting Group. Workshop on the Business ExcellenceModel, London, 1998.

British Quality Foundation. Voluntary Sector Guide to Self-Assessment.London: British Quality Foundation, 1997.

Brunsson, N., and Olsen, J. P. The Reforming Organisation. London:Routledge, 1993.

Charities Evaluation Services. Quality in Charities: Using the Excel-lence Model to Promote Self-Assessment in Charities. London:Charities Evaluation Services, 1998.

Clarke, J., and Newman, J. The Managerial State. London: Sage, 1997.Conti, T. Organizational Self-Assessment. London: Chapman and Hall,

1997.Darvill, G. Organisations, People and Standards. London: Association

of Directors of Social Services and the National Institute of SocialWork, 1998.

Di Maggio, P., and Powell, W. W. “The Iron Cage Revisited: Institu-tional Isomorphism and Collective Rationality in OrganizationalFields.” American Sociological Review, 1983, 48, 147–160.

Drucker, P. F. The Drucker Foundation Self-Assessment Tool: Partici-pant’s Workbook. San Francisco: Jossey-Bass, 1998.

Farley, T. Practical Quality Assurance System for Small Organizations.London: Charities Evaluation Service, 1997.

Garvin, D. A. “How the Baldrige Award Really Works.” Harvard Busi-ness Review, Nov.-Dec. 1991, pp. 80–93.

Ghobadian, A., and Woo, H. S. “Characteristics, Benefits, and Short-comings of Four Major Quality Awards.” International Journal ofQuality and Reliability Management, 1994, 13 (2), 10–44.

Harris, M. “Clarifying the Board Role: A Total Activities Approach.”In D. R. Young, V. A. Hodgkinson, R. M. Hollister, and Associates,Governing, Leading, and Managing Nonprofit Organizations: NewInsights from Research and Practice. San Francisco: Jossey-Bass, 1992.

Harris, M. “Exploring the Role of Boards Using Total ActivitiesAnalysis.” Nonprofit Management and Leadership, 1993, 3 (3),269–281.


The models“work” to the

extent that theyfacilitate new

dialogues aroundissues felt to beimportant, newways of makingsense of those

issues, and newplans for howthey may beaddressed

Hassell, T. The NGO Finance Charities Internal Audit Checklist. NGOFinance, 1998.

Kearns, K. P., and others. “Why Nonprofit Organizations Are Ripe forTotal Quality Management.” Nonprofit Management and Leadership,1994, 4 (4), 447–460.

Kirkpatrick, I., and Martinez-Lucio, M. (eds.). The Politics of Qualityin the Public Sector. London: Routledge, 1995.

Lowndes, V. “Varieties of New Institutionalism: A Critical Appraisal.”Public Administration, 1996, 74, 181–197.

Martin, L. L. Total Quality Management in Human Service Organiza-tions. London: Sage, 1993.

McTiernan, A. Report on PQASSO Pilot Carried Out for the Commu-nity Council of Devon for CRISP. Exeter, England: CommunityCouncil of Devon, 1998.

Morgan, C., and Murgatroyd, S. Total Quality Management in the Pub-lic Sector. Buckingham, England: Open University Press, 1994.

National Center for Nonprofit Boards. Self-Assessment for NonprofitGoverning Boards. Washington, D.C.: National Center for NonprofitBoards, 1999.

Powell, T. C. “Total Quality Management as Competitive Advantage:A Review and Empirical Study.” Strategic Management Journal,1995, 16 (1), 15–37.

Powell, W. W., and Di Maggio, P. J. (eds.). The New Institutionalismin Organizational Analysis. Chicago: University of Chicago Press,1991.

Quality Standards Task Group. A “White Paper” on Quality in the Vol-untary Sector. London: Quality Standards Task Group/NCVO, 1998.

Rowntree, D. The Manager’s Book of Checklists. London: Pitman,1996.

Samuels, M. Towards Best Practice: An Evaluation of the First Two Yearsof the Public Sector Benchmarking Project, 1996–98. London:Cabinet Office of Her Majesty’s Government, 1998.

Seddon, J. The Vanguard Guide to Business Excellence. Buckingham,England: Vanguard, 1998.

Senge, P. M. The Fifth Discipline: The Art and Practice of the LearningOrganization. New York: Doubleday, 1990.

Stern, G. J. The Drucker Foundation Self-Assessment Tool: ProcessGuide. San Francisco: Jossey-Bass, 1998.

Weick, K. E. Sensemaking in Organizations. Thousand Oaks, Calif.:Sage, 1995.

Yin, R. K. Case Study Research: Design and Methods. Thousand Oaks,Calif.: Sage, 1984.


Documents

What Happens When Nonprofits Use Quality Models for Self-Assessment?