Policy Studies Journal, Vol 15, No 1, September, 1986

SECTION TWO- MEASUREMENT OF HUMAN RIGHTSPRACTICES

MEASURING HUMAN RIGHTS:PROBLEMS AND POSSIBILITIES^

Andrew D. McNitt

We are now able to systematically anaiyze humanrights abuses on a worid wide basis Amnesty International(since 1974>, Freedonfi House (since 1973), and the StateDepartment (since 1979) have prepared annual reports ofthe status of human rights for most of the nation statesof the worid Eventually we wiii be able to perform timeseries anaiyses of human rights abuses Yet the measure-ment of human rights abuses is a difficult task There areconceptuai, ideoiogicai, and measurement probiems whichmust be considered by anyone who wishes to work withthese data

CONCEPTUAL DIFFICULTIES

One of the most serious probiems with existingstudies of human rights is the iack of conceptuai ciarityThe quantitative analysis of human rights requires us tomake normative as well as empirical judgments We makenormative judgments when we decide what constitutesabuses ot human rights We make empirical judgmentswhen we decide how to measure respect for those rights

While It IS possible to measure the extent to whichgovernment practices conform with expiicitiy stated setsof vaiues, confusion is caused by the refusal to explicit-ly state those values, the lack of consensus on thosevalues, and the premature use of summated measures ofthose vaiues The failure to expiicitiy define respect forhuman rights is most evident in studies (Strause andClaude, 1976, Henderson, 1982) which borrow existingscales from earlier works (Gastil, 1973) Since thesescales grow out of a specific ideological position, careshould be taken to alert the reader to the normative and

72 Andrew D McNitt

empirical implications invoived when using a particuiarmeasure

The lack of a consensus on the meaning of respect forhuman rights further complicates measurement Respect forhuman rights, although part of the definition of democ-racy, IS not equivalent to democracy Democratic statesoccasionally fail to respect human rights and nondemo-cratic states do not uniformly abuse those rights Someeariy efforts to measure democracy deal with that conceptaimost exclusively in terms of popular sovereignty andeiectorai choice (Outright, 1963, Neubauer, 1967, Outrightand Wiiey, 1969), whiie others mix in some aspects ofrespect for human rights (Fitzgibbon, 1956, Fitzgibbon andJohnson, 1961, Smith, 1969, Jackman, 1973, Boiien, 1980)Unfortunateiy, even when information on respect for humanrights IS included in the definition of democracy, sub-scales relating to respect for human rights are usuallynot retrievable from the literature Oonsequently, exist-ing measures of democracy are inadequate measures ofrespect for human nghts

A second area of disagreement involves the conflictbetween economic and political rights There is a tendencyfor those on the left to include economic rights, income,health, and physical conditions in their definition ofhuman rights, and to distinguish those nghts from anarrower set of procedural nghts While I would not wantto excuse national states from their social responsibil-ities by defining those responsibilities away, I preferthe narrower definition We live in a conservative societywhich defines human nghts m legal and political termsOonsequently if we hope to influence that society we alsomust adopt a similar restrictive definition

A third area of controversy involves the judgmentsabout the context within which human nghts abuses occurWhile there is some general agreement as to the standardsof respect for human nghts required from stable govern-ments, there is less consensus on how nations at warshould behave The stresses of both external and internalconflict often lead to government violations of individualiiberties How should we evaluate these violations? If wetry to take circumstances into account we have to developa second set of standards for evaiuating beiiigerents A

Comparative Human Rights Policies 73

second set set of standards for belligerents might pro-hibit torture and arbitrary killings while allowing forsome degree of censorship and control over movementWhile this approach would work for international con-flicts. It would be difficult to apply in cases otinternal war internal war is probiematic because it ishard to determine preciseiy when such conflicts begin ifwe take the narrower view that any violation of liberty isa violation of human nghts we simpiify our measurementproblems, but end up having to classify many acts of selfdefense as vioiations Still, if the focus of inquiry isthe abuse of human nghts, then it is probably best toignore questions of justification and concentrate on thephenomenon whenever and wherever it occurs

A fourth area of controversy involves the distinctionbetween individual and group nghts Shouid the definitionof human nghts inciude nghts that properly belong togroups' Such nghts as the nght to organize, to withholdservices, and to act coiiectiveiy, although most commonlyassociated with tabor unions, are also important to otherkinds of secondary groups However we choose to classifythese nghts, a decision must be made for inclusion orexclusion That decision is particularly important whendeveloping a singie aggregate measure of "human rights" inwhich case we must be careful not to include statisticallyand conceptuaiiy disparate measures

One way to avoid the problems caused by globalmeasures of human nghts is to concentrate on a smallsubset of core nghts (Forsythe and Wiseberg, 1979) whichare usually defined as freedom from torture, freedom fromimprisonment for the mere expression of a belief, andfreedom from political execution The concept of corehuman rights is more universally acceptable than some ofthe legal and procedural definitions which have theirongins m Western legal practices Even when we use theconcept of basic human rights, however, we stili cannotreach a consensus For example. Amnesty Internationaltreats aii appiications of the death penalty as vioiationsof human nghts Aithough this practice simpiifiesmeasurement, it interjects yet another value conflict intothe process Further, if we restnct our analysis of humannghts to basic human rights we then ignore both less

74 Andrew D McNitt

extreme abuses of human rights and violations of poiiticalrights

The lack of a consensus as to the meaning of humanrights makes any attempt to use a single summated measureof those rights a risky endeavor Too often authors haveused summated scales without analyzing the interrelation-ship between items (Gastil, 1978; Strause and Claude,1976, Henderson, 1982). A preferable approach would be tobegin either by examining individual practices or, ifscaiing is desirable, by using seating techniques such asfactor analysis or Gutman scaling to establish unidimen-sionality before employing summated measures AmnestyInternational's (1983) report on killings, Seymour's(1979) accounting of prisoners of conscience, and Nixon'sstudies (1960, 1965) of freedom of the press are examplesof studies which follow this first strategy of restrictedanalysis Cingranelh and Pasquarello's (1985) study whichuses factor analysis to dimensionalize human rights abusesIS an example of the second type of scalar analysis Vaiueconflicts need not destroy our ability to study humanrights abuses so long as we resist the temptation to reiyexclusively upon summated measures of those rights indi-vtduai variabies such as the number of executions, the useof torture, and the number of prisoners of conscience canbe analyzed in their own right and subscales can bedeveloped to measure a variety of different categories ofhuman rights

EMPIRICAL DIFFICULTIES

A number of empirical difficulties aiso compiicateOKxr efforts to measure human rights abuses The degree towhich measurement difficuities are a problem depends uponthe uses we wish to make of the data it is easier to findinformation about human rights abuses in singie societies.Consequentiy, there has been an abundance of case studiesof human rights abuses. Unfortunately, it is impossible toadequately test theories about either the causes or conse-quences of human rights abuses m general when workingwith data from a single society Cross sectional data setsare harder to develop because of the difficulty of findingcomparable measures from the same period of time for a

Comparative Human Rights Policies 75

large number of societies We would, however, substan-tially improve our understanding of human rights abuses ifwe did more cross sectionai studies Still, even crosssecttonai data has its iimitations The cyclic nature ofhuman rights abuses and the desire to discourage govern-ment human rights abuses means that we need to developcross sectionai time series data sets Yet the desire tohave cross sectional time series data makes it even harderto obtain data and requires more precise measurement thanIS characteristic of either the case study or cross sec-tional models alone

The biases of informants and reporting groups, lackof information about abuses, and problems in attributingresponsibility for abuses also makes it difficult todevelop reliable measures The biases of informants andreporting groups affect the coMection of information intwo ways First, mdividuai informants usuaiiy are aiiiedvtfith one side or the other in a political disputeExaggerated reports of atrocities are not an uncommonpropaganda technique, consequentiy, such reports shouidnot be taken at face vaiue Fortunateiy, major reportinggroups such as Amnesty internationai and the State Depart-ment's human rights reports do make some effort to verifyaccounts of human rights violations Unfortunately, theseand other organizations have their own rather distinctpoints of view and have been criticized for their biases(iVlcCamet, 1981, Scobel and Wiseberg, 1981) Amnesty Inter-national writes from a liberal perspective. Freedom Housefrom a conservative perspective, and the State Departmentfrom a more compticated perspective The State Departmentmust be frank enough to satisfy Congress, diplomaticenough not to overly offend American aiiies, and at thesame time responsive to the President's wishes Probabiythe best technique for dealing with the biases of thereporting groups is to use two or three of their reportsin combination when coding in an attempt to neutraiizesome of the bias or—as Stohl, Carlton and Johnson (1984)did—to construct three separate scales (one for eachsource) to use m the analysis tn addition, it is alsoadvisable to base our measurements on the descriptiveportions of these reports and to ignore the more subjec-tive measures Estimates of the extent to which human

76 Andrew D McNitt

rights abuses are changing, global categorizations ofsocieties as free or unfree and nonquantitative compari-sons between societies should be avoided because of theirgreater vulnerability to reporter group bias

It IS often difficuit to obtain rehabie informationabout human rights abuses because of the probiems ofcollecting information about closed societies, difficui-ties in determining the number of abuses, and the pro-vincial location of some of those abuses Closed societieshave harassed and imprisoned individuais for attempting tomonitor human rights abuses, others have made it difficultfor international monitoring groups to obtain access toinformation, and still others, because of governmentcontrol over the locai media and restrictions on travel,make it difficult to obtain even the simplest politicalinformation

Information on the actual number of abuses committedIS even more difficult to obtain when a government wishesto disguise the extent of those abuses This problemvanes by type of abuse Information on official execu-tions, especialiy when they are publicized as an examplefor others, is relatively easy to obtain It is muchharder to get an accounting when government forces, eitherwith or without official sanction, murder individuals andhide their bodies Estimates of the number of peopietortured are even more difficuit to obtain Aimost allpolice departments, especially in the process of makingarrests, brutalize peopie The problem is to determine ifan individual act of brutaiity is part of a pattern ofabuse or only an isolated incident Victims of abuse areafraid to come forward and most of the time only a smallpercentage of the more notorious incidents come to lightOne aiternative to estimating the exact number of abusesIS to use either a rough ordinai scaie or to code anysubstantial pattern of abuses as a violation

The location of human rights abuses also influencesour ability to detect them Amnesty International andFreedom House do not prepare reports on ali countries TheState Department's human rights reports now have sectionson aii members of the United Nations, but the entries forsome of the smaiier states are extremely brief The loca-tion of abuses within a country aiso has an impact

Comparative Human Rights Policies 77

Reports of human rights abuses are less precise when theydeai with abuses committed outside of the major urbanareas or in speciai security zones In such countries asColumbia and Peru there are persistent reports of greaterhuman rights abuses by government forces operating inrural as opposed to urban areas (State Department,1983 510, 675)

It IS not always easy to attribute responsibility forabuses Individual disappearances may be a result ofgovernment policy or the responsibility of decisions madeby independent members of a security service or the resultof potential victims fleeing a dangerous situationFinally, either the deliberate or unintentional failure ofgovernments to enforce the law can contribute to theoutbreak ot vigilante activities and communal violenceThese types of extragovernmental violence are extremelydestructive of human rights, but are less directly theresponsibility of government officials than is violencewhich results from the actions of government agents

SOURCES OF INFORMATION

Expert opinion, events data analysis, and organiza-tional reports have all been used to develop measurementsof human rights abuses Summated expert opinion with allIts problems has been used by Fitzgibbon and his asso-ciates (Fitzgtbbon, 1956, Fitzgibbon and Johnson, 1961) toperiodically measure democracy m Latin America, and byNixon (1965) to measure freedom of the press for 117nations In all of these cases, panelists were given alist of countries and asked to rate them either in generalterms or along a number of dimensions which were latersummed to produce a single scale score This technique,although roughly accurate, leaves too much to chanceGeneral impressions, even expert ones, are only a crudemethod of classifying nation states The problems ofreliability and validity are particularly severe when thenumber of experts is much smaller than the total number ofstates Expert opinion tends to rely on overall impres-sions of a society and hence is probably relatively insen-sitive to cyclic changes in performance

78 Andrew D McNitt

Events data analysis has also been used to measurehuman rights abuses In most cases this analysis is basedon an accounting of government actions as reported in theNew York Times Index Taylor and Hudson (1972) developed ameasure of government sanctions from the NYT Index whichcounts ati government actions designed to controi domesticopponents as sanctions The accounting is avaiiabie foraii nation states, but mixes vioient and nonvioient eventstogether and has not been updated iVlarkus and Nesvold(1972) used the NYT index to create a Gutman scale ofgovernment coerciveness for ten countries The Markus andNesvoid scaie is an interesting effort to measure thecoercive aspects of government actions, however, it hasnot been extended in time or in scope beyond a fewsocieties and ignores legal and procedural violations

Organizational reports are the most promising sourcesof information on human rights violations Amnesty Inter-national, Freedom House, and the United States StateDepartment ali prepare annual human rights reports Eachreport contains a brief country report on most of thenation states in the world which summarizes the status ofhuman rights in that society during the year

Amnesty Internationai is the most respected of thereporting groups Amnesty International is a worid wideorganization which endeavors to act as an advocate forpolitical prisoners It has established a reputation forfairness and consistency in its reporting techniquesAlthough founded in 1961, its reports are difficult toobtain prior to the mid-1970s and even today do not reporton alt nationai states

Freedom House, m addition to preparing a series ofcountry reports, also rates nationai states as free orunfree Freedom House has been severeiy criticized for itsconservative biases and for providing reiativeiy iittleinformation about how data is coliected (Saizberg, 1979)Even so, the country reports are consistent and theorganization has criticized both right and ieft wingregimes

The State Department's annual human rights reportsare the most detailed and complete of the annual reportsThe entries relate specific incidents, quote conclusionsof nongovernmental monitoring groups, and are two or three

Comparative Human Rights Policies 79

times as extensive as those of the other monitoringgroups The first two reports, 1975 and 1976, however,were criticized by members of Congress who then officialiyturned to the Library of Congress for aiternate informa-tion As a result of this fiasco, substantial changes weremade in the way the reports were prepared These changesimproved the scope, quality, and independence of thereports after 1979 (Scobei and Wiseberg, 1981) A reviewof the 1983 reports in 22 nations which was conducted byAmericas Watch, Helsinki Watch, and the Lawyer's Committeefor Internationai Human Rights (1983) concluded that "Forthe most part distortions that appear in the CountryReports for 1982 are not so great as to prevent policymakers from grasping the essentials of the human rightssituation " (7), however, " information aboutabuse in some friendly countries, but by no means ali[SIC] IS presented m such a way to dimmish its impact orexpiam it away" (5)

Cingraneiii and Pasquareiio (1985) compared the StateDepartment country reports with Amnesty International'sannual report for 1980 for 100 randomly selected countrieson use of torture and arbitrary imprisonment They foundsubstantial agreement, 81 percent and 77 percent respec-tively, in these two categories On the other hand, Stohl,Carlton and Johnson (1984) found what they considered tobe a relatively low correlation between the State Depart-ment and Amnesty internationai when they compared theirratings of 23 Latin American states for 1976 and 1981However, Stohl, Carlton and Johnson's anatysis of therelationship between human rights abuses and U S foreignaid produced statistical relationships which were essen-tiaiiy the same when respect for human rights was measuredwith State Department, Amnesty International, and FreedomHouse data

Changes in presidential administrations have aisomfiuenced these reports Under the Reagan Administrationthe reports have been expanded to include informationabout human rights abuses caused by insurgent groups aswell as governments, and to limit the presentation ofeconomic and social information (Americas Watch, 1983)

The State Department reports are a frustrating sourceof information, too good to be ignored and yet flawed

80 Andrew D McNitt

enough to require us to be cautious when using them As asource, the human rights reports are probably good enoughto be used to construct a data base if we ignore thesubjective portions of the reports, use a iarge number ofcases, and look oniy at general patterns of behavior Formore specific tasks, such as monitoring human rightsabuses in specific societies, we shoutd use the countryreports only in conjunction with other sources

Finally, a number of students of judicial behaviorand criminal procedures have begun to look at convictionrates, sentencing practices, executions, and the extent ofprocedurai protections provided by triai courts m theUnited States Unfortunately, oniy a very smaii amount oftruly comparative studies of legal procedures and judicialbehavior have ever been attempted (Baar, 1976) If wecould obtain a reasonably compiete international data set,this kind of information would offer us much greaterability to precisely analyze human rights abuses Short ofthis comparative information about the use of the deathpenalty,^ imprisonment rates and conviction rates would beof interest even though somewhat tangentiaiiy related toactual abuse of human rights

CONCLUSION

Just as the natural sciences do not refuse to studyimportant phenomena simply because they are difficult tomeasure, so too the social sciences should not abandon thestudy of human rights simply because they are difficult tomeasure If we are careful, we can deai with most of thenormative and empirical problems well enough to producesubstantial improvements m our knowledge of how govern-ments treat their citizens

The normative problem of disagreement over themeaning of human rights can be solved by agreeing todisagree Human rights should be treated as a muiti-dimensionai phenomena By separateiy anaiyzing specificaspects of respect for human rights and basing our scaleson the statisticai interrelationships between thesevariables we can avoid the necessity of developing aSingle definition of the concept

Comparative Human Rights Policies 81

The empirical problems &re real, but thanks to thedevelopment of systematic annual human rights, reports areless severe today than in the recent past So long as weare willing to accept more modest leveis of precision wehave the data to identify major causal relationships andtrends

NOTES

The preliminary work for this project was completed during aNational Endownient for the Humanities Summer Seminar on Theoriesof Collective Action I would like to thank Sidney Tarrow and theNational Endowment for the Humanities for their encouragennent andsupport I would also like to thank Andrea Bonnicksen for hercomments

Amnesty Internationai published a 1979 report on the deathpenalty This report provides a comparative description of theapplication of the death penalty bul does not give an accouniingof execution rates

REFERENCES

Americas Watch, Helsinki Watch, and The Lawyer's Committee for Inter-national Rights 1983 Review of Human Rights Practices for 1982

York Americas Watch)

Amnesty International 1979 The Death Penalty (London Amnesty Inter-national Publications)

Amnesty International 1983 Political Killings by Governments(London Amnesty International Publications)

Baar C 1976 'Judicial Behavior and Comparative Human RightsPolicy' Pp 353^381 m R Claude ed, Comparative Human Rights{Baltimore Johns Hopkins University Press)

Bolten, K 1980 'Issues in Comparative Measurement of Democracy'American Sociological Review 4513) 370-390

Cmgranetli, DL and TN Pasquareiio 1985 'Human Rights Practicesand the Distribution of U S Foreign Aid to Latin American Coun-tnes " American Journal of Political Science 29(3) 539-563

Cutnght, P 1963 'National Political Development Measurement andAnalysis ' American Sociological Review 28(2) 253-264

82 Andrew D McNitt

Cutnght, P and J Wiley 1969 "Modernization and Political Repre-sentation 1927-1966" Studies m Comparative InternationalDevelopment 5(1) 23-44

Fitzgibbon, R 1956 "A Statistical Evaluation of Latin AmericanDemocracy" Western Political Quarterly 9(3)607-619

Fitzgibbon, R and K Johnson 1961 "Measurement of Latin AmericanPolittcal Change" American Political Science Review 55(3)515-526

Forsythe, D and L Wiseberg 1979 "Human Rights Protection AResearch Agenda" Human Rights Quarterly 1(4) 1-25

Gastil, R 1973 Freedom in the World Political Rights and CivilLiberties (New York Freedom House)

Henderson, C 1982 "Military Reginies and Rights in Developing Coun-tries" Human Rights Quarterly 4(1) 110-123

Jackman, R 1973 "On the Relation of Econonnic Development to Demo-cratic Performance" American Journal of Political Science17(3)611-621

Markus, G and B Nesvold 1972 "Governmental Coerciveness andPolitical Instability" Comparative Political Studies 5(July)231-244

McCamet, J 1981 "A Critique of Present Measures of Human Rights andAn Alternative" In V Nanda, R Scarrett and G Shepherd, eds,Global Human Rights (Boulder, CO Westview Press)

Neubauer, D 1967 "Some Conditions of Democracy" American PoliticalScience Review 61(4) 1002-1009

Nixon, R 1960 "Factors Related to Freedom in Nationai PressSystems" Journalism Quarterly 37(1) 13-28

NiKon. R 1965 "Freedom of the World's Press A Fresh Approach WithNew Data" Journalism Quarterly 42(1) 13-14

Saizberg, J 1979 'Monitoring Human Rights Violations How Good inthe Inform at ton''" In P Brown and 0 MacLean, eds. Human Rightsand US Foreign Policy (Lexington, MA Lexington Books)

Scobel, H and L Wiseberg 1981 "Problems of Comparative Research onHuman Rights" In V Nanda, J Scarrett, and G Shepherd, eds.Global Human Rights (Boulder, CQ Westview Press)

Seymour, J 1979 "Indices of Political Imprisonment" Universal HumanRights 1(1)99-103

Comparative Human Rights Policies 83

Smith, A 1969 "Soctoeconomic Development and Democracy A CausalAnalysis ' Midwest Journal of Political Science 13(1)95-125

State Department 1983 Country Reports on Human Rights Practices(Washington, DC U S Government Printing Office)

Stohl M, D Cartton and S Johnson 1984 'Human Rights and USForeign Assistance From Nixon TO Carter" Journal of PeaceResearch 21{3) 215-216

Strause, J and R Claude 1976 Empirical Comparative RightsResearch Some Preliminary Tests of Developmental Hypotheses" InR Claude, ed. Comparative Human Rights (Baltimore JohnsHopkins University Press)

Taylor, C and M Hudson 1972 World Handbook of Political and Socialindicators (New Haven CT Yale University Press)