Statistical Practice in Bureaucracies

Statistical Practice in BureaucraciesAuthor(s): Margaret E. MartinSource: Journal of the American Statistical Association, Vol. 76, No. 373 (Mar., 1981), pp. 1-8Published by: American Statistical AssociationStable URL: http://www.jstor.org/stable/2287032 .

Accessed: 10/06/2014 10:28

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

American Statistical Association is collaborating with JSTOR to digitize, preserve and extend access to Journalof the American Statistical Association.

http://www.jstor.org

This content downloaded from 195.34.78.29 on Tue, 10 Jun 2014 10:28:52 AMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=astata

http://www.jstor.org/stable/2287032?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


Statistical Practice in Bureaucracies MARGARET E. MARTIN*

1. INTRODUCTION

More than 140 years ago a few men met in Boston to form what shortly became known as the American Sta- tistical Association. Article II of its first constitution ex- plained the purpose succinctly: "The objects of the society shall be to collect, preserve and diffuse Statistical information in the different departments of human knowledge" (American Statistical Association 1940).

We have come a long way since then. It may be useful, however, to refresh our minds concerning that original purpose. Government statistics bureaus are among the more prominent organizations specifically charged to collect, preserve, and diffuse statistical information. Stat- isticians in the private sector who apply statistical methods to problems in science, in industry, in medicine, for exainple, are sometimes puzzled by the different interests of their colleagues in government bureaus of statistics. They ask questions such as, Why does not the Office of Federal Statistical Policy and Standards have more staff trained in statistical methods? Why do not more statisticians become heads of government statistics agencies? I shall not try to justify the present state of affairs, but I should like to comment on some of the distinguishing characteristics between statistical practice in government and outside government. I have two objectives. One is to draw attention to some of the important current issues in federal statistics. Another is to emphasize the importance of certain statistical practices beyond those commonly considered a part of statistical science.

Statistical methods applied to problems in the laboratory, on the agricultural plot, in the manufacturing control process may not differ much whether the sponsor is a government, a university, a corporation, or an individual scientist. But statistics viewed as a product, that is, data collected and disseminated for the use of others, does differ in important ways from statistics considered as a science or a methodology. It is to this part of government statistics that the rest of this paper is confined.

Persons trained in statistical methodology think of statistics as a scientific discipline, but most of the rest of the world uses the term to mean orderly arrays of data, an end product of statistical activity-sets of numbers, tables, or graphs. Many of us are ambivalent and move back and forth, using the discipline-oriented concept when talking about the qualifications of a particular statistician for filling a particular post, and using the product-

* Margaret E. Martin is Senior Research Associate, Committee on National Statistics, National Research Council, 2101 Constitution Ave., Washington, DC 20418.'

oriented definition when referring to the functions and activities of government statistics bureaus. (For further discussion, see Kruskal 1978, Healy 1978.)

What is different about statistics considered as product rather than as method? Let us look at the process of producing and summarizing large data sets for the use of others, the principal function common to large government statistics bureaus. In one sense, producing statistics as an end product is a narrower concept than applying statistical methodology across a whole range of problems; any one bureau of statistics or similar organization usually focuses on only one class or kind of subject matter area. In another sense, it is broader. It encompasses not only statistical methodology as a tool, but the whole gamut of activities that must be performed in producing statistics for the use of others-planning, collecting, analyzing, and disseminating data. The practice of many of these functions is not based primarily on statistical science or methodology, but is an art based on a mixture of intuition, experience, and judgment, as well as scientific evidence or procedures-in other words, the practice of a profession as well as the application of a scientific discipline (Hartley 1980). Statistical methodology is a part, but only a part, of the process. I believe these state- ments are descriptive of any application of statistics, but that they are more evident components of statistical activity in organizations whose primary function is data collection and compilation of general purpose statistics than in other areas of application.

When I joined the federal statistical system in 1943 there was, of course, major emphasis on the production of statistics to guide the war effort. More basic than this temporary practical use of detailed data, however, was the widely and even passionately held belief that the success of our economic and social systems depended to a significant degree on the quality and widespread availability of statistical information about the operation of these systems. Economic statistics had recently been overhauled and expanded during the depression decade, and there was hope that the state of society could simi- larly be quantified (Committee on Government Statistics and Information Services 1937). The development of the national accounts and the Employment Act of 1946 added further impetus. At that time, the users of the information, the decision makers, were viewed as innumerable, anonymous individuals using impartial information to make a multitude of decisions on matters of importance

? Journal of the American Statistical Association March 1981, Volume 76, Number 373

Presidential Address

i



2 Journal of the American Statistical Association, March 1981

to them. This view contrasts with the view of statistics collected for a particular purpose to fit the needs of a particular problem with an identified decision maker. Both views may reflect reality, of course, sometimes for the same data set.

During the last three decades the awareness of statistics as a tool of central decision making has grown, accom- panying the expanded use of statistics in planning, operating, and evaluating many government programs, in monetary and fiscal policy, and in regulatory activities. The growing reliance on statistics in central decision making should be gratifying to statisticians; the results of our labors are being used. It also imposes on government and the statistical profession great responsibilities in supplying relevant and accurate information.

Increased reliance on statistics in decision making has also led to a view of statistical information as power. This, in turn, has created some uncertainty over the proper location of statistics-gathering organizations within the federal government. On the one hand, the statistics must be relevant to the problems perceived by society; on the other, they must be neutral with respect to partisan solutions. If the statistics bureau is too close to policy planning and analysis, it runs the risk of becoming partisan. If it is too insulated from policy considerations, the statistics bureau may be perceived and treated as a rou- tine production operation, a mere numbers factory, a view all too prevalent among administrators. For this reason, as well as to improve the quality of the data themselves, heads of statistics bureaus seek to establish subject matter analytical units within their staffs. Such units help to identify priorities (see National Research Council 1976), interact with technical statisticians, deal with policy analysis groups established elsewhere in their departments on questions of concern to departmental decision makers, interact with the research community and other outside users, and provide a channel through which all these interests have an impact on the data collection operation. Without such internal analytical ca- pability, planning functions are likely to suffer.

In the following I shall identify what appear to me to be important characteristics of the process of providing data for the use of others. The list is by no means ex- haustive. I have chosen topics either because of their important effects on the activities of statistics bureaus or to direct attention to aspects not normally considered a part of statistical science that might benefit from additional attention from highly trained statisticians.

2. PRIORITIES

A critically important function of an organization producing statistics is to set priorities. What statistics must be collected? Develop new data or improve the accuracy or efficiency of an ongoing series? Use more resources in analysis, in testing the quality of current operations, or in disseminating results more widely? Needs, technical feasibility, and costs must be considered. The choices

are myriad, the guideposts infrequent, and the advice of different stakeholders (or users) conflicting. A large federal statistics bureau is at some distance from the users of its data, from internal government decision makers (sometimes even those in its own department), to say nothing of other users such as the Congress, other levels of government, researchers in universities and elsewhere, and the general public. A conscious effort must be made to keep up with the needs of these various groups and to resolve conflicting views into a sensible program. The difficulties are great. The process is not well understood even by those who are trying to determine priorities, still less by others. An example from my own experience at the Bureau of the Budget may illustrate the difficulties.

You will recall the enormous pressure to speed up our space program and land a man on the moon following Sputnik. The space agency had proposed a greatly increased budget and the budget examiner responsible for reviewing this request asked me what the matter was with the statistical program at the National Science Founda- tion. It appeared that one group of critics was charging that if the budget request were to be approved it would require the employment of half the trained physicists in the country; another group countered that the program would only need 10 percent of the physicists. The Na- tional Science Foundation could not tell the examiner the correct figure overnight. He thought that the coordinating statistical office, in which I was employed, should work with the Foundation to develop a more relevant, policy- oriented program of statistics to be better prepared for the next urgent demand for information. He was unable to tell me, however, or to obtain from his colleagues even a tentative list of issues for which statistics would be needed in the future. Similar examples could be found concerning every statistics-producing unit in the federal government.

I draw two conclusions. One is that policy makers and other users frequently judge the quality of a statistics program on the basis of what types of statistics are produced-whether they answer the important immediate questions, rather than on the quality of those statistics. The other conclusion is that statistics bureaus are unlikely to get immediate and easy answers about either present or future needs from the people for whom the statistics are compiled. Subject matter analysts and practicing statisticians must become involved on an interacting basis. As pointed out earlier, experienced bureau chiefs want analysis units within their own staffs-units that by en- gaging in substantive studies can learn the virtues and deficiencies of the existing information; can deal on a continuing basis with policy analysts and decision makers outside the bureau; and can keep abreast of external research. Such activities are those most likely to point to needs for new or different information. Such units should have explicit responsibility for participating in setting priorities.

In setting priorities, as in much else of its planning and design undertakings, a federal statistics bureau operates



Martin: Statistical Practice in Bureaucracies 3

within a decentralized, but loosely coordinated, statistical system. Coordination is intended to make the separate data collections of different bureaus more useful (because the results are designed to be aggregated or to be analyzed in concert) or to make them more efficient (by eliminating duplication or unnecessary differences in definitions or by arranging for access to independent sampling frames, for example). A recently completed review of the coordinating function recommends centralizing and strength- ening the coordinating office and, among other improvements, would explicitly assign to that office responsibility for leadership in improving the application of statistical methodology in federal data collection activities (Bonnen 1980). Legislation to accomplish this has recently been introduced in the Senate.

A fairly recent development, initially motivated by the American Statistical Association, has been the organization of the Committee of Professional Associations on Federal Statistics (COPAFS), which now includes 12 member associations. The general objectives are to increase the involvement of the professional associations in issues of general importance in federal statistics and to monitor the changing situation as it affects the federal statistical product relevant to the user needs of the associations. Sufficient support has become available to establish an executive office for COPAFS in the early fall of this year. It is expected that in the future there will be increasing interaction between the federal statistical system and the professional community representing many aspects of special user interests, among which will, no doubt, be the setting of priorities.

3. CONCEPTS

Another major activity of statistics bureaus is the definition of concepts, an activity that is both difficult and extremely important in much of federal multipurpose data collection. Both economic and social theory have devoted insufficient attention to defining units of analysis in terms appropriate for data collection. Unlike many of the experimental sciences, concepts are seldom rigor- ously tested under laboratory conditions. A well-known example is the case of employment and unemployment. Macroeconomics deals with "labor" and the price of labor. If one wants to measure labor, should one measure the number of persons at work, or jobs held, or hours spent at work? There are uses for each of these measures. It took economists and statisticians working together for nearly a decade during the thirties, first to agree on basic concepts, and second to develop reasonably satisfactory and mutually compatible definitions of employment and unemployment, definitions that were workable in household surveys and that with only minor changes are still being used today, though not without continued questioning.

Another, and as yet unsettled, example is the concept of ethnicity and the problem of its definition in censuses and surveys. Social theory suggests no easy answer,

usually assuming self-identification. A variety of different definitions have been used in practice: place of parents' birth, mother tongue, language spoken in the home, origin or descent. The current census asks, "Are you a . . ?" and then provides a check list. Some are dubious that this form of the question will produce useful results and have argued that the questions on birthplace of parents, despite their more limited relevance, are to be preferred in view of their greater objectivity and presumed reliability.

Ethnic origin is but one of numerous characteristics and classifications that require continuing dialogue, ex- perimentation, and testing, with both statisticians and subject matter specialists making contributions. These issues do not occur only in the social sciences, but range across the life and physical sciences as well. The National Center for Health Statistics devoted much effort to develop a widely applicable concept of disability, finally arriving at a definition of conditions limiting activity. Those compiling statistics on skin cancer are plagued by the difficulty of distinguishing between new and recurring cases. Oil and gas production statistics are riddled with definitional problems. And so it goes. It is not sufficient for the statistician to say that such concepts and definitions are not his or her business, that these questions belong exclusively to the subject matter specialist.

4. DESCRIPTIVE STATISTICS

The multipurpose statistics collected by government statistics bureaus are primarily descriptive statistics. The immediate occasion for their collection is usually a practical one, not the search for scientific truths. Neverthe- less, these descriptive statistics are frequently incorporated into scientific theories or assist in the development of new knowledge. The population censuses, originally prescribed for the apportionment of congressional seats, have, together with the birth and death statistics developed to promote public health, formed the basis of the science of demography. The Census of Manufactures and our statistics of international trade, designed to support the infant economy of a new country, were the forerun- ners of an enormous array of economic data that are used not only for current monitoring of the economy but also to test economic theories and are incorporated into econometric models to analyze relationships. Weather statistics provide some of the raw data for the atmospheric sciences. Thus opportunities do arise for the development of new knowledge in connection with what may appear to be mundane data collection activities.

The processes of collecting, summarizing, analyzing, and disseminating data, furthermore, afford ample opportunity, given sufficient resources, for the application of statistical methodology and the development of new statistical techniques. Illustrative of these opportunities are survey sampling theory, estimates of statistical error, the handling of incomplete data in surveys and in analysis, matching or linking of data sets, the design of experiments used in testing concepts, definitions, or collection meth-




ods, and various analytical devices, such as simulation models and the research on methods of seasonal adjust- ment now being furthered under a joint ASA-Census Bureau program.

5. ESTIMATES OF ERROR Government statisticians have long worked on prob-

lems of statistical error and have been particularly suc- cessful in telling users about sampling error. Attempts to describe biases and response error, and thus total error, have been a more recent development. A recent effort at quantifying nonsampling error is the report "An Error Profile: Employment As Measured by the Current Pop- ulation Survey" (U.S. Office of Federal Statistical Policy and Standards 1979). Statisticians need the cooperation and assistance of subject matter specialists to identify and describe the sources and magnitudes of response error and bias. Yet the concept of bias measured only in a statistical sense, as a difference between the expected value of the estimate and the true value being estimated, seldom satisfies the subject matter specialist, who is in many cases more concerned with the relevance of the expected value of the quantity measured to the concept that fits his or her theory or problem. This may be particularly true of analysts who are not collecting their own data but are using data collected for other purposes, as so frequently happens with multipurpose economic and social data. I recall hearing a statistician ask an economist why economists paid so little attention to the error structure in the data they used. The economist replied that in her view, conceptual relevance of the data was 85 percent of her problem, and error but 15 percent. Even though statisticians cannot incorporate such concerns in error formulas, they can be sensitive to issues of relevance in joint explorations of error measurement. John Tukey called attention to these issues in his recent paper on the statistician's responsibility for both accuracy and relevance (Tukey 1979).

Because data produced by federal statistics bureaus frequently are national in scope, and therefore refer to the same universe, differences in series that ostensibly measure the same or very similar phenomena are glaringly obvious. This can be acutely embarrassing to the pro- ducers of the data. If two different bureaus are involved, each is likely to point to all the reasons the other's estimates are weak-reasons that sound sensible, are frequently based on supposition, rather than scientific evidence, and seldom tell the whole story. I worked at one time with a small interagency group that put together employment and unemployment estimates from different sources into a single public release. To take the employment estimates, for instance, each was produced monthly, referred to the same time period, used the same classifications (with some exceptions), but was based on a different operational definition of employment. One was a probability sample of persons with a job, obtained from households with an estimated sampling error and (in com- parison with the other) with considerable month-to-

month sampling variability. The other was based on reports of filled jobs from employers, and thus included double counting of persons holding more than one job, was not a probability sample, but was subject to much less monthly variation. Adherents of the former series pointed to its proper probability design, overlooking the bias of the population undercount; adherents of the latter pointed to the presumed accuracy of employer reports compared with household surveys, and the lower variability, overlooking the biased survey design. Meanwhile, the public wanted to know what was happening to employment each month. Even if estimates of total error had been available for each series, I doubt if that question could have been answered definitively (although no doubt our uncertainty would have been greatly reduced), since each estimate of error would have been couched in terms of the definitions appropriate to its own frame. Lacking estimates of total error, the interagency group writing the monthly release tried to understand the strengths and weaknesses of each series, to explain reasons for glaring differences in month-to-month changes (not always pos- sible), and to give the public an overall view of what was happening to some ill-defined, underlying concept of employment as reflected by two different measurements. These series are still being announced in a combined release, although the responsibility is now that of a single bureau (see "The Employment Situation," press release issued monthly by the U.S. Bureau of Labor Statistics).

The point I should like to make, however, is that the existence of two ostensibly similar series can be a boon as well as a bane to statistical enterprise. Differences in level and change can call attention to problems, especially to nonsampling errors and, if taken seriously, can lead to improvements in one or both series. Efforts to rec- oncile the two employment series, for example, led to many improvements, extensions, and additional analy- ses-improvements that have provided a wealth of additional information on the labor force. Attempts to ex- amine relevance, as well as nonsampling error, might well start with comparisons between other such closely re- lated series.

6. PRESSURES FOR MORE Another characteristic circumstance of much federal

data collection is the insistent pressure for more. Once statistical information is produced, it is generally used. And once used, pressures start building for more-more detail, greater frequency, more prompt availability. If a series is issued quarterly, it is wanted monthly, if it is national in scope, it is wanted for regions, or states, or all standard metropolitan statistical areas, or even every county (3,000) or every political entity (39,000). The Census Bureau has been faced several times during the past decade with the task of making population and in- come estimates for 39,000 units as updates from the 1970 Census for general revenue-sharing purposes. Half of these 39,000 units have a population of less than 1,000. The Bureau of Labor Statistics is currently required to




estimate employment and unemployment monthly for 5,000 areas. Other statistics bureaus face similar demands for state and local data.

The quality of such efforts is limited not only by resources-budgets, statistical talent, and other inputs to the statistical operation-but also by the burden imposed on respondents. Although much of the government's paperwork burden is not for statistical purposes-tax collection, regulatory activities, provision of benefits for which applications must be filed-large-scale statistical surveys must consider the burden imposed on suppliers of data and the mounting resistance to paperwork. Such resistance is particularly strong in the case of small business organizations. Statistical surveys, furthermore, may depend on tax or regulatory or other administrative reporting programs to establish universes, provide sampling frames, or reduce the burden of reporting for small units. The economic censuses, for example, use a few items of information from tax returns rather than collecting du- plicate information directly from large numbers of very small firms.

Economizing on the burden of answering forms is thus a federal objective (see Commission on Federal Paper- work 1977). Exchange of information among statistics bureaus for statistical purposes only, and with strict confidentiality for individual reports, is one device that can economize on both resources and burden (President's Commission on Federal Statistics 1971; ASA Committee on Privacy and Confidentiality 1980). Another device for reducing burden is the joint collection of information by federal and state statistics bureaus. Other devices have been sought, such as the use of synthetic estimates in which ratios from a current national survey are applied to less current state or local information to develop current state or local estimates (Purcell and Kish 1980). The search for new estimating and analytic methods should continue to help relieve the pressure for ever-increasing detail.

Much of the demand for geographic detail arises from legislation. The most historic use is prescribed in the Consitution, the use of population census counts to determine the number of representatives from each state and to fix the boundaries of congressional districts. More recently, Congress has specified increased detail in legislation to ensure that funds are distributed to localities in accordance with its intent, or has required that information needed for planning or evaluating the success of various programs is available. Formulas are incorporated into laws for the distribution of funds in more than 100 programs in which population data are used (U.S. Con- gress 1975). In other programs, information such as the unemployment rate is used to trigger the starting and stopping of benefit programs in local areas. Some work has been started on ways to tailor allocation formulas or trigger mechanisms so that statistical errors will have less effect on the results (U.S. Office of Federal Statistical Policy and Standards 1978), but much more needs to be done. Uses of statistics for fund allocations or triggering

mechanisms bring pressures for increased accuracy and additional detail. As Nathan Keyfitz has pointed out (1979), such uses of statistics assume certainty rather than the uncertainty with which statisticians are accus- tomed to deal. As he noted, certainty can only be assumed by convention, conventions that are necessary because otherwise dollars could not be parceled out or programs started and stopped by formula.

The 1980 Census will, of course, be used for much of the fund distribution in the next few years. In the past, the convention has been implicit that census figures are correct for such purposes. Census statisticians, evaluating the completeness of the decennial censuses, have called attention to an undercount of the population that is relatively more serious for blacks and probably other mi- nority groups. These estimates of undercount, derived by methods of demographic analysis, are not available for small areas; even state estimates of the undercount are uncertain. Now, because of increasing awareness of the undercounts, their presumed uneven impact on different localities, and the greatly expanded use of census statistics in allocation formulas, the Bureau of the Census faces the troublesome question of whether adjustments should be made in Census data, and if so, in what fashion (see National Research Council 1978 and U.S. Bureau of the Census 1980).

Statisticians should note that this problem is not confined to enumerating the population or to fund allocation. In private contracts as well as in federal regulation, fund allocation, or administration, many other statistics, rang- ing from wage rates, prices, and corporate financial figures to emissions of pollutants and estimates of air quality, are used as if they were without error.

Adjustments are used relatively freely in some estimates, such as the national accounts. Here the data are controlled by an accounting framework and missing pieces estimated and inadequate pieces adjusted by the economists and statisticians who prepare the estimates. This practice, as well as the fact that the national accounts are pieced together from hundreds of different data sources, has effectively prevented any meaningful estimates of the error structure of the accounts. There are obviously very different views and practices regarding adjustments in different data sets.

Statisticians should be concerned, not only with estimating statistical error, but with methods for dealing with uncertainty in situations that require an assumption of certainty in- the statistics. How far, for example, should a statistics bureau go in making adjustments, under what circumstances, on what basis, with what justification? These are questions that statisticians have normally shunned or considered the province of others, possibly because of the policy implications.

7. COUNTING Statisticians engaged in collecting multipurpose data

are perceived by the rest of the world as engaged in counting-an operation often regarded as the most elementary




numerical process, not only by the public but also by other statisticians. Even though the mathematical design for collecting information may be simple, the practice of collection is almost invariably difficult. Errors, and the opportunity for making errors, abound. One need not point only to the population undercount. During the 1973 energy crisis there were problems with data on imports of oil, reflecting difficulties of assigning imports to the same time periods in different sets of records. The U.S. and Canada have undertaken a detailed, laborious inves- tigation attempting to match total imports of the one country with exports from the other, and could not resolve all discrepancies despite years of good faith effort. Employer reports of employment and wages to tax agencies differ from statistical surveys even when the tax records are the mailing list sources for the surveys. In addition to sampling error, likely causes are different reference periods, incompatible definitions of units, classification problems, errors of omission, double counting, and response error.

8. ANALYSIS

Since statistics bureaus collect data primarily for the use of others, some believe it is unreasonable to expect them to engage in substantive analysis. They may fear that such analysis might distort data collecting priorities or delay public dissemination of the data. Nevertheless, a criticism frequently expressed is that statistics bureaus do not do enough analysis. I agree with the latter views. Of course, there is usually some analysis. For example, the development of models and projections may be considered one form of analysis-population projections, input-output tables, econometric models, or seasonal ad- justment procedures have all been developed at least in part by statistics bureaus. Currently, several longitudinal surveys are being supported by federal agencies, with funding for analysis provided primarily to outside researchers. Development of new analytic techniques for efficient analysis of longitudinal surveys may possibly emerge.

I believe that more analysis by statistics bureaus, whether by their own staffs exclusively or in cooperation with others, would lead to improvements in the base data, to a better understanding of priorities, to better documentation of the data for use by others, and possibly to new knowledge as the result of the analysis. The close tie between analysis and data improvement is not generally understood and, indeed, is not clearly documented. Budget authorities and congressional appropriations committees are unaware of the importance of analysis in leading to improved data and also may fear that analysis would not be politically neutral. The research community is, on the whole, considerably more interested in obtain- ing the data for its own independent use than concerned about building up analytic strength in federal statistics bureaus. Nevertheless, there are signs of increasing analysis in some statistics bureaus and some examples of

cooperative analysis between such bureaus and independent researchers, and I hope these can be encouraged.

9. DISSEMINATION

Statistics bureaus spend much time on methods of dis- tributing their products effectively. The theoretical statistician writes a report for a research journal and typically leaves it up to others to seek out and use the results of his research. The statistical consultant has direct contact with the users of his advice. But the statistics bureau, at a greater distance from users and operating on public funds, should feel a responsibility for making data available in convenient forms and for explaining how the data were compiled and outlining their weaknesses and strengths. Many of you may feel that few statistics bureaus have gone far enough in this direction.

Efforts to improve dissemination are being made, however. For example, microdata samples for public use have been prepared much more frequently in recent years, and rather than the last results to be disseminated, are now much more likely to be near the first. Some bureaus provide computer access to large arrays of aggregated data. Some statistics bureaus have instituted occasional conferences of users to promote the use of their data for research and to obtain feedback from users. Documentation remains a problem, one of concern to all applied statisticians on which much work should be encouraged in the future.

Dissemination to policy makers and the general public is frequently accompanied by descriptive comment and graphs, calling attention to noteworthy facts. It is to be hoped that the recent resurgence of interest in tabular presentation and in graphical techniques (Beniger and Robyn 1978; Fienberg 1979) will lead to improved practice. How the news media adopt and adapt such news releases is still another problem. It may be too much to expect headline writers to understand measurement error. Just recently, a headline in the Washington Post read, "Jobless Rate Edges Up to 7.8 Per Cent." The one- tenth of one percent change reported in the unemployment rate was less than one standard error of consecutive monthly change. Even so, this was an improvement over a headline 20 years ago that read, as I remember, "Gov- ernment Says Unemployment Unchanged, But Actually It Increased by One-Tenth of a Point."

There is another type of dissemination-of information about the methods and practices developed and used in statistics bureaus. This is commonly an undernourished activity. More needs to be done, not only in encouraging reporting of methodological research in the Journal of the American Statistical Association and in other professional journals, but in describing methods and procedures that might be useful to others but are not sufficiently original for inclusion in JASA. I hope that more opportunities will be afforded in the future for such publication. I look forward to the day when the American Statistical Association will find a way in its publication program to




recognize the needs of statisticians and their colleagues engaged in data collection, compilation, and dissemination-not by changing JASA, of whose worldwide re- known we can all be proud-but by one or more additional publications. If statisticians as professionals seek a wider application of statistical methods in data collection, in other scientific disciplines, and in management and administration, whether public or private, we may need to devote more effort to communicating with nonstatisti- cians in interesting and beneficial ways.

10. CONCLUSION I have touched on some of the concerns of statistics

bureaus in producing data for public use, intentionally skipping over most purely statistical science concerns. -In describing some of the circumstances in which statisticians practice in statistics bureaus, my purpose has been to encourage more interaction with other disciplines and more attention by statisticians to problems of concept, public policy, or practice-concerns that they may initially view as beyond their bailiwick. In this review of statistics as a process of data collection, compilation, and dissemination in federal bureaus, I have made a number of recommendations that I briefly summarize here.

I urge acceptance of the proposal for a strengthened central coordinating office for federal statistics; I support legislation that would permit exchange of data among statistical bureaus with suitable confidentiality safe- guards; and I recommend strong internal analysis staffs in statistics bureaus.

One thread that runs through the other recommendations is the need for strong and continuing interaction between statisticians and subject matter specialists-in setting priorities within statistics bureaus, in developing concepts and definitions, in exploring relevance as well as the measurement of error, and in improving methods of disseminating data.

Furthermore, I have suggested that statisticians, whether in or outside statistics bureaus, can make contributions to methods for dealing with the uncertainty inherent in statistics in such practical applications as formula allocations of funds; can develop improved methods of data collection and analysis to meet the increasing demands for additional detail; need to devote much more attention to problems of documentation; and should be encouraged to publish more descriptions of methods and procedures for nontechnical as well as technical audiences.

Statisticians who follow these suggestions will tend to concentrate on one or a few areas of application. In this way, they can obtain sufficient background to help fill the wide gaps between methodology and practice. Such steps may be deemed to run counter to the view that statistics is not specific to subject matter, that statisticians should move easily from one field of application to another, and thereby spread new methodology more widely and more quickly. I believe these views need not

be in conflict. Both types of statisticians are needed. In my opinion, the training of statisticians in statistical sciences gives insufficient weight to the need for statisticians interested in data collection problems (see Federer 1978; Committee on Training of Statisticians for Industry 1980). Many of the statisticians now engaged in data collection have moved to it from other disciplines. There might be a two-way flow. Indeed, if my analysis is correct, there is a sizable meeting ground where it is difficult to distin- guish the statistician from the subject matter specialist.

REFERENCES AMERICAN STATISTICAL ASSOCIATION (1940), Minutes of meet-

ings of December 11, 1839, and February 5, 1840, reprinted in "Pro- ceedings of the Centenary Celebration, 1839-1939," Journal of the American Statistical Association, 35, Pt. 2, 297-301.

ASA COMMITrEE ON PRIVACY AND CONFIDENTIALITY (1980), "Business Directories: Findings and Recommendations of the ASA Committee on Privacy and Confidentiality," The American Statisti- cian, 34, 8-10.

BENIGER, JAMES R., and ROBYN, DOROTHY L. (1978), "Quan- titative Graphics in Statistics: A Brief History," The American Stat- istician, 32, 1-11.

BONNEN, JAMES T. (1980), "Improving the Federal Statistical Sys- tem: Report of the President's Reorganization Project for the Federal Statistical System," Statistical Reporter, 80-8, 197-212.

COMMISSION ON FEDERAL PAPERWORK (1977), Final Summary Report, Washington, D.C.: U.S. Government Printing Office.

COMMITTEE ON GOVERNMENT STATISTICS AND INFOR- MATION SERVICES (1937), Government Statistics, A Report of the Committee on Government Statistics and Information Services (Bull. 2), New York: Social Science Research Council.

COMMITTEE ON TRAINING OF STATISTICIANS FOR INDUS- TRY (1980), "Preparing Statisticians for Careers in Industry: Report of the ASA Section on Statistical Education Committee on Training Statisticians for Industry," The American Statistician, 34, 65-75.

FEDERER, WALTER T. (1978), "Some Remarks on Statistical Edu- cation," The American Statistician, 32, 117-121.

FIENBERG, STEPHEN E. (1979), "Graphical Methods in Statistics," The American Statistician, 33, 165-177.

HARTLEY, H. 0. (1980), "Statistics As a Science and As a Profes- sion," Journal of the American Statistical Association, 75, 1-7.

HEALY, M.J.R. (1978), "Is Statistics a Science? Journal of the Royal Statistical Society, Ser. A, 141, 385-393.

KEYFITZ, NATHAN (1979), "Information and Allocation: Two Uses of the 1980 Census," The American Statistician, 33, 45-56, including comments by Harold Nisselson and Harry V. Roberts and rejoinder by Nathan Keyfitz.

KRUSKAL, WILLIAM H. (1978), "Statistics: The Field," in Inter- national Encyclopedia of Statistics (Vot. 2), eds. William H. Kruskal and Judith Tanur, New York: The Free Press, 1071-1098.

NATIONAL COMMISSION ON EMPLOYMENT AND UNEM- PLOYMENT STATISTICS (1979), "Comparing Data From Different Sources," in Counting the Labor Force, Washington, D.C.: U.S. Government Printing Office, 193-205.

NATIONAL RESEARCH COUNCIL (1976), "Setting Statistical Prior- ities," Panel on Methodology for Statistical Priorities, I. Richard Savage, Chairman, Committee on National Statistics, Washington, D.C.: National Academy of Sciences.

(1978), "Counting the People in 1980: An Appraisal of Census Plans," Panel on Decennial Census Plans, Nathan Keyfitz, Chairman, Committee on National Statistics. Washington, D.C.: National Acad- emy of Sciences.

PRESIDENT'S COMMISSION ON FEDERAL STATISTICS (1971), Federal Statistics (Vol. 1), Washington, D.C.: U.S. Government Printing Office, 153-155.

PURCELL, N.J., and KISH, L. (1980), "Postcensal Estimates for Local Areas (or Domains)," International Statistical Review, 48, 3-18.

TUKEY, JOHN W. (1979), "Methodology and the Statistician's Re- sponsibility for Both Accuracy and Relevance," Journal of the American Statistical Association, 74, 787-793.




U.S. BUREAU OF THE CENSUS (1980), Conference on Census Un- dercount, Proceedings of the 1980 Conference, Washington, D.C.: U.S. Government Printing Office.

U.S. CONGRESS (1975), Federal Formula Grant-in-Aid Programs That Use Population As a Factor in Allocating Funds, compiled by the Library of Congress, Congressional Research Service, for the use of the Subcommittee on Census and Population, House Committee on Post Office and Civil Service, Committee Print 94-6, 94th Con., Ist sess., Washington, D.C.: U.S. Government Printing Office.

U.S. OFFICE OF FEDERAL STATISTICAL POLICY AND STAND- ARDS (1978), Statistical Policy Working Paper 1, Report on Statistics for Allocation of Funds, Washington, D.C.: U.S. Government Print- ing Office.

(1979), Statistical Policy Working Paper 3, An Error Profile: Employment As Measured by the Current Population Survey, Wash- ington, D.C.: U.S. Government Printing Office.



Documents

Statistical Practice in Bureaucracies