Download pdf - Whatever happened to bureaucracy? And why does … · Whatever happened to bureaucracy? And why does it matter? ... Whatever happened to bureaucracy? 2 ... My plan is to talk about

1

Whatever happened to bureaucracy?

And why does it matter?

Prepared for the Arc pattillo masterclass, Wellington, 16-17 October 2006. Based on the ”Practitioner keynote” presentation

at the Industrial and Organisational Psychology conference, Marriott Resort, Gold Coast, Queensland, 30 June to 3 July 2005,

supplemented with material from complexity workshops.

Bob Dick [email protected]

We are in the middle of a prolonged shift in our social systems, locally and globally. This is a “singularity”, an event with such dramatic effects that we cannot predict longer-term outcomes.

Here I identify two aspects of the shift. The first is the collapse of bureaucracy. Already begun, this is transforming our social systems towards more team-based structures. The other, barely evident, is the gradual failure of rational problem solving and decision making for some purposes.

Each of these changes threatens disruption. However there are opportunities here. The opportunities are especially relevant for those who understand people and organisations. We can make important contributions, theoretical and practical, if we are willing to make the most of the possibilities.

Keywords: beliefs without reason, bureaucracy, change, chaos, complexity, Cynefin, decision making, future, globalisation, human resource functions, hybrid organisations, networks, organisations, planning, problem solving, singularity, social systems, structure, teams, virtual teams


2

In futures studies a singularity is an event with effects so large and varied that they obscure everything that follows. There are so many possibilities that the future becomes uncertain. Actually, futurists like Ramez Naam (2005) talk about the singularity. By this they mean the time at which artificial intelligence exceeds human intelligence. They locate it mid-century — 2050 or thereabouts. (I should mention that futurists aren’t much better at prediction than anyone else.)

It’s unlikely we’ll wake up one morning to find a dramatically changed world run by computers. Singularities aren’t point events. The artificial intelligence singularity is already in train: a computer beating a chess master is a symptom. The effects of the singularity may still be playing out a millennium from now. Singularities can be very slow and very gradual.

Probably that’s why we’ve barely noticed that we’re in the middle of another singularity. I think it’s an important singularity, and a complex one. That’s why I’m addressing it here. Globalisation is involved as both result and cause.

My plan is to talk about aspects of this singularity, this turning point for the world — aspects which offer some enticing opportunities. Some are especially relevant for those of us who work with people and social systems. In particular I’ll address two aspects. The first is the impending failure of bureaucracy. The second, less apparent, is the impending failure in some situations of rational planning, problem solving and decision making. I’ll follow these with a discussion of the opportunities I believe they provide to people like consultants and facilitators.

I begin with bureaucracy.

The rise and demise of bureaucracy

Let me first say something about what I intend “bureaucracy” to mean. I’m using it as description, not criticism. Max Weber (1921) thought it was the rational way to


3

structure large organisations. In some circumstances I agree with this characterisation of it, defined (among other aspects) by the structural notions of hierarchy and regimentation. People doing similar jobs are grouped together. For example, when I worked as an electrical draftsperson, I was in a large room with all of the other electrical draftspeople and engineers. At the same time, the people who make decisions are separated from those whose task it is to make the decisions work. Decision making is centralised.

The result is a familiar triangular shape. This is how many people still visualise the shape of organisations and other social systems.

It’s uncertain when bureaucracy first arose. A reasonable estimate would place it at the beginnings of agriculture. That’s conventionally dated to about 10,000 years ago, though Steve Taylor (2005) thinks agriculture preceded bureaucracy by several thousand years. He may be right. In any event it was unlikely to have happened quickly and it was a long time ago.

My guess is that once bureaucracy was developed, bureaucracy and agriculture may have coevolved. Agriculture is likely to have benefited from the specialisation that bureaucracy allowed, for instance into warrior and agricultural classes. Bureaucracy would presumably have been favoured by the stability permitted by agriculture.

We do know that bureaucracy worked well enough to become the standard structure for almost all social systems. This included not just industrial, commercial and government organisations, but also families and tennis clubs, for example. In my early youth a typical family exhibited clearer differences in rank and role than you’d find in many families today.

Early signs of the collapse of bureaucracy have been with us for several decades. They could be seen in the academic literature of the 1960s. In many respects this earlier


4

literature is worth reading today. Less specialised than much current journal literature it is often very informative and easy to understand and use.

Some examples. Joan Woodward (1965) compared the effectiveness of manufacturing companies in southern England. Less bureaucratic organisations were found in less stable environments. Tom Burns and George Stalker (1961) found similar differences between industries. The fast changing industries like plastics and electronics moved towards the looser structures that Burns and Stalker labelled organic. In their important work on structure within organisations, Paul Lawrence and Jay Lorsch (1967) found much the same.

This all makes sense. Imagine that you’re someone at the workface of a bureaucratic organisation. The organisation is rational and procedure driven. You notice some important change in the organisation’s environment — a shifting market, for instance. In such an organisation you have no authority to change your behaviour. So you communicate the news upward. In time, new procedures work their way back down to you. If the environment has changed again by the time this happens, the new procedures probably won’t work.

For reasons I’ll address shortly, the push towards less bureaucratic structures has continued. We’ve reached a point where many organisations have moved towards more team-based structures. Others are in the process of doing so, or are thinking about it. Many of them find it more difficult than they imagine. Some colleagues and I were asked to evaluate a previous structural change from conventional to team based structures. In about half the teams we evaluated there may have been a change to the organisation chart. But the “teams” continued to function as they always had. We weren’t able to identify any teams where the restructure was a resounding success.

The organisation underestimated the difficulty of the change. I’ll argue below that most organisations do.


5

In theory — and in the better organisations, in practice — we are able to use the new structures effectively at team level. The work of team leader and team becomes the task for the team as a whole. The team leader becomes a “boundary rider”, managing coordination outside the team. In structure the team shifts ...

from this ... to this ...

If I were still doing electrical design work, it’s almost certain that I’d spend my time in a series of project teams. Each team would assemble around a particular design task. Each team would contain an architect, an electrical designer, a civil designer, a structural designer, a quantity surveyor, and so on. As each project was completed we’d move to different projects.

You’ll have noticed that the two structures diagrammed above are quite different. Team leader and team member roles and responsibilities have changed substantially. The required skills have also changed. That’s why the shift is often more difficult than people anticipate.

For an entire organisation the shift is even more problematic. It’s usual for the number of organisational levels to be reduced. The organisation is flattened. It is also interlaced with myriad project teams, meetings, committees and working parties. It seems that they are needed to keep the organisation functioning.


6

There is a puzzle here, though. Why is the shift so difficult? In some respects we’re just moving back towards pre-bureaucratic structures.

There are good reasons to believe that we’re a tribal species. Even if you don’t accept Robin Dunbar’s (1992) argument about cortex and tribal size, the summary he provides of group size amongst humans and other primates is persuasive. You might think that modern organisational structures would suit a tribal species. Work teams offer more design flexibility. Workers can be offered more autonomy, variety and learning within more meaningful jobs. Among others, Fred Emery and his ANU colleagues (Emery and Emery, 1975) have argued this for some years. Robert Rehm (1999) has said it again more recently. For most people, bureaucracies were not the most satisfying places to work.

But if that’s so, why don’t these new structures work more effectively more often? That question could be of interest to consultants and facilitators.

I think our genetic inheritance does equip us well to make the most of the new structures at team level. The problem is to be found, I think, in our social inheritance: our memes. Several thousand years is long enough for us to be well enculturated into thinking about structures as bureaucratic structures. We’ve developed many beliefs that support such structures.


7

In addition, many thousand years is time enough for many of these supporting beliefs to sink out of awareness. The result is that many of our beliefs are not easily accessible to conscious examination. They are “beliefs without reason”. To this, too, I’ll return.

Many of our current organisations are hybrids, part bureaucracy and part something else. Bureaucratic structures, driven by procedures, require employees to be obedient. That was part of Weber’s (1921) recipe for organisational success. On the other hand team based structures require innovation and responsibility. We might therefore anticipate that employees in hybrid structures experience tension between such conflicting requirements.

It’s difficult to anticipate the resolution to this conundrum. Networks of networks are possible, perhaps, if Dee Hock (1999) is right. They seem to work well at Visa. Sociocracy as implemented by Gerard Endenburg (1998) also looks promising. It combines team-based structures, decision-making by consent, and autonomy within limits for each team.

But after all, I’m arguing that we are in a singularity. It’s not surprising that the end-point is mostly still obscure at this stage.

The rise and partial demise of rational planning and problem solving

Now, rational planning, decision making and problem solving ...

We seem to be in an earlier phase of this second aspect of the singularity. I’m referring to the threatened failure under some circumstances of our rational methods for planning and problem solving. Under some circumstances rational methods still work. But often they don’t, as Henry Mintzberg (1994) has argued.


8

Explanations for this are to be found in the literature on complexity. Increase the complexity and turbulence of a phenomenon and the relationship between cause and effect becomes unclear. It has been called the “butterfly effect”. The incident that triggered the interest in chaos and complexity provides an illustration.

In the early 1960s Edward Lorenz modelled weather on a small computer (Lorenz, 1972). As you can imagine, at that time it wasn’t a very sophisticated computer. However, with a limited number of interconnected variables it behaved on paper very like real weather systems in nature.

On one occasion he noticed an interesting pattern, which he decided to study. He keyed in the data again to watch how the pattern developed. It didn’t.

Initially the program followed a path similar to the one it followed earlier. Then, gradually, it diverged. Eventually it became quite different. As you know, a digital computer is a deterministic machine. Run the same program on the same variables several times. Each time the results are the same.

At one level the explanation was simple enough. On the first run the data were typed in to six decimal places. In his haste, on the second occasion Lorenz only keyed in three decimal places. You’ll note that the data were still the same to an accuracy of a thousandth of a unit. To me that seems like reasonable accuracy for weather variables. I wonder if weather forecasters enjoy such accuracy.

Lorenz (1972) reported the results in a paper entitled “Predictability: does the flap of a butterfly’s wings in Brazil set off a tornado in Texas?” The “butterfly effect” became part of our vocabulary.

It’s not that a butterfly can deliberately start a tornado. It’s that some complex systems are sensitive to initial conditions. Very small changes in initial conditions can lead to


9

very large changes in eventual outcomes. But the point is that you can’t tell which initial changes will produce which results.

This is important. Under some circumstances you can’t tell what the effects of a small change will be. To say it differently you can no longer predict the effects resulting from a particular cause. You can’t always tell what the outcomes of your actions will be. This is no trivial matter. Imagine what it would be like if you couldn’t predict the results of your actions.

That means that there are circumstances where rational planning and problem solving and decision making don’t work.

Think of a continuum ranging from simple to chaotic. Imaging turning on a tap very slowly and smoothly. From predictable drops of water, evenly spaced, it becomes a steady stream. At higher pressures and volumes still it becomes like a thing alive, turbulent and unpredictable.

It now appears that just before a system turns chaotic it passes through a zone now labelled complex. Some of the literature refers to this as “the edge of chaos”. In some respects it turns out to be a more populated space than might be imagined. It’s where many social systems are to be found.

It’s useful to think of this continuum in terms of cause → effect relationships. In the simple zone the cause → effect relationships are obvious and known. In the complicated zone they can be known in principle, but require study and research. Rational planning, decision making and problem solving still work, however.


10

In complex or chaotic zones the cause → effect relationships become unpredictable. It’s no longer useful to think in terms of cause and effect. That means rational planning and problem solving are no longer appropriate.

I can illustrate this with a metaphor from Stafford Beer (1975). He asks you first to imagine a social system as a number of ball bearings connected by springs. Then imagine giving one of the ball bearings a small jiggle. If there are few ball bearings and relatively few interconnections the whole system will jiggle briefly in response. It will then quickly settle down.

Now imagine many ball bearings, richly interconnected. Your initial small jiggle may reverberate though the system for some time. As the effects work their way through the many sequences of springs, parts of the system may jiggle more than the initial ball bearing did. It’s possible that the ball bearing that started it all will itself be shaken around greatly.

Globalisation is creating systems with more nodes — more ball bearings — and more interconnections between them. It’s likely that the side-effects of much of our past actions are still working their way through our global ecology. Possibly the energy crisis will begin to decouple the world by making transport very expensive. In other respects, though, our actions seem to be increasing the interconnections rather than decreasing them.

That’s not to say that everything is complex. Much of the time we can predict the likely results of our actions. When cause → effect relationships are known or knowable, rational planning and problem solving will continue to be effective. However, when

• almost everything affects almost everything else, or

• there are multiple feedback loops, some time-lagged, or

• some of the variables change often or continuously


11

then you may be in the complex zone — on the edge of chaos. If so, expect difficulties.

Craig Reynolds’ (2001) computer simulations demonstrate that unpredictable behaviours can emerge when each of a flock of simulated birds follow just three rules: match alignment, avoid crowding, and steer for the centre.

I’m not suggesting that the unpredictable behaviour of human systems can be so simply explained. When people who exercise choice are added to the system then unpredictably can increase further.

• When we are expert in a field we make sense of complex information by seizing the first pattern which fits. As Gary Klein and his colleagues have shown we seldom seek the best fit pattern (Klein, 1997).

• We have such a need to make sense of experience that we find patterns where none exists. We loosen our criteria until a pattern is found. We then search for evidence to confirm our guess.

• We “see” cause-effect relationships which may not exist. We tend to impute motives to others. We then act as if our assumptions were factual.

• As David Snowden (2004) has argued we have multiple identities or “roles”. Depending on which identity is active we behave differently at different times.

• By act of will we can decide to act in more or less ordered ways. We can be obedient or difficult, for example.

These qualities have implications for how we facilitate problem solving and decision making. The less predictable the situation, the more useful it is to take these qualities into account. For example:

• Non-experts may be better at spotting an unusual pattern. We can capitalise on this by mixing novices with experts. We can use informal networks and novel processes. People drawn from different areas of expertise can challenge one anothers’ premature solutions.

• We can encourage the identification of multiple patterns. We can do what we can to create a climate where difference and dissent can be expressed constructively. In


12

particular we can encourage participants to look for disconfirming evidence and exceptions.

We can encourage people to test their assumptions, especially about others’ motives, before acting on them.

Physical layout, novel settings, and careful instructions can do much to activate different identities.

The more we make allies of participants the more likely they are to seek to cooperate with us and with each other.

In contrast with this, think of the way many of our organisations try to behave.

It is also instructive to compare bureaucratic behaviour to the description by Karl Weick and Kathleen Sutcliffe (2001) of “high reliability” organisations. These are organisations that can’t afford error: aircraft carriers, nuclear power plants, firefighting crews and the like.

Weick and Sutcliffe describe such organisations as mindful in the way they ...

• carefully analyse failures and partial failures in a no-blame way

In these organisations you are rewarded for admitting you made a mistake, not punished for the mistake.

• accept and try to understand the complexity of the situations they face — they don’t oversimplify

They know that their world is too complex to be understood fully. They are aware that an unavoidable catastrophe may be lurking around the next corner.

• pay attention to the operational aspects of what they do as well as the strategic aspects

When they are making strategic decisions they consider the operational aspects. People responsible for operations are present to speak for themselves.

• develop high resilience and flexibility, for quick recovery

It’s too late to pay attention to this in a crisis. They devote ongoing attention to developing resilience and encouraging everyone to take responsibility.


13

• defer to expertise, wherever it is to be found.

Expertise outranks rank. People defer to the person with the expertise, understanding and experience. Ideas are judged on their merits.

The Cynefin Centre (e.g. Snowden and Stanbridge, 2004) has developed an approach to problem solving and decision making which can cope with complex situations. They capture their approach as the “Abide” model, which I can summarise as the following five recommendations for working with complex situations:

provide multiple A ttractors have a flexible B oundary activate appropriate I dentities encourage D issent and diversity provide an appropriate E nvironment

This accords well with facilitation strategies I’ve developed through experience — many facilitation settings are complex. The box below provides an illustration.

In partial summary ...

Let’s pause for a moment and recollect the ground we’ve covered.

There are occasions when bureaucratic structures work well, though they are becoming fewer. There is a general move towards more team-based structures and more flexible organisations. However, there is some doubt about which structures will be effective. Teams may fit our genetic make-up as a tribal species. They don’t always work as well as expected.

As globalisation creates more complex systems we can expect bureaucratic structures to become less and less effective. At the same time, systems that are complex (in the technical sense) are less predictable. Rational planning and problem solving can be expected to lose much of their effectiveness.


14

Facilitating complex situations

• It’s not possible to predict what will work. Therefore, encourage participants to develop multiple initiatives and options. Monitor those initiatives closely for success or failure. Accept that trial and error is required.

• Participants may bring inappropriate bureaucratic behaviours that interfere with creativity and flexibility. Reduce the behaviour by defining appropriate limits, preferably by negotiation. Keep the limits flexible.

• Participants are capable of fitting into many different roles. They may not know which of them are most useful. I can encourage appropriate roles, for instance through careful instructions and negotiation.

• Participants are likely to move to premature decisions and solutions. These solutions are likely to draw on their own training and experience at the cost of other possibilities. Anything I can do to encourage diversity and dissent will therefore be useful. For example I can aim for high diversity within groups. I can introduce naive or novice participants. I can encourage cross-fertilisation between groups

• I can create an environment, physical and psychological, that conveys useful and appropriate signals to participants about the style and purpose of the task.

The potential contribution of consultants and facilitators

Times of change challenge our beliefs and our strategies. At the same time they offer opportunities to those who are willing to seize them. Consultants and facilitators are well placed to make the most of some of those opportunities.

It will be seen that some of the opportunities are new. Others are more a reworking of traditional approaches. Others may need only better integration. I list some of them below. Finally, I address the importance of beliefs in sustaining the conventional and undermining the new.


15

First, structure. As bureaucratic and procedure-driven systems collapse, organisations will be forced to move to other structures. These attempted changes will bring a range of challenges in their wake. For example

• managing the hybrid organisations, half bureaucracy and half something very different

• helping team members develop the skills and orientations that enable effective team decision making and problem solving

• helping team leaders abandon much of their control of their team, giving their attention instead to facilitating the team, and managing upward and outward rather than downward

• rediscovering our tribal commitment to good relationships while learning to manage diversity and conflict

• coping with the growing use of virtual teams and temporary project teams.

Note how often our traditional organisational behaviours are supported by ingrained habits and entrained beliefs developed over generations — beliefs without reason. A bureaucratic mindset has been burned into our memes. It’s often inaccessible to conscious examination.

This is also true of the cultural changes that will be needed to support the structural changes. For example, as procedures can no longer keep up with the rate of change, there are some changes we can anticipate:

• the use of guiding principles rather than detailed procedures

in those situations when the results of our actions are not predictable, detailed procedures are of little use; principles allow people to pursue outcomes with flexibility

• the worth of values-based rather than instrumental organisational allegiance — people who identify with the organisation because they approve of what it does and how it does it


16

but then it’s no longer enough to recruit people with the right skills and experience; it is also necessary to give attention to the values they bring to their work.

We see around us instances of the shrinking half-life of careers, skills, solutions, strategies, and the like. What if the rate of change increases further? If we don’t decouple the world there is reason to expect that it will. It may be that the energy crisis will decouple the world for us by making transport very expensive. But if the world stays closely coupled, an increasing rate of change will be accompanied by:

• a growing need for creativity and innovation at all levels of organisations

• learning which becomes increasingly an unavoidable and partly individual dimension of change

• a need for greater tolerance of ambiguity on the part of all of us.

The Australian economist John Quiggin (1997) has provided some evidence that employees have experienced increased work intensity. If so, the changes of the near future will add to the stresses that work intensity and job insecurity have generated. As the need for learning is increasing, the time to engage in it is being reduced.

Beliefs without reason

I’ve already said that there are well-embedded beliefs and behaviour that support traditional structures and strategies. In my experience, when organisations move people to a new structure, those people take their old beliefs and behaviours with them. Elliot Aronson (1976) hints at much of the difficulty:

“As long as I know why I believe x, I am relatively free to change my mind; but if all I know is that x is true—and that’s all there is to it—I am far more likely to cling to that belief, even in the face of disconfirming evidence.” (p. 299-300)


17

To say this differently, we may be working with people who carry over beliefs from earlier ways of structuring organisations or solving problems. Our current structures and solutions may depend on these beliefs changing. The beliefs may be held without reason. Our temptation will be to use rational debate and explanation to bring about the change.

Reason is ineffective at changing beliefs without reason. In effect, such people have been inoculated against evidence. Without a change in beliefs, however, new behaviours are unlikely to be sustainable.

And of course we are handicapped by our own beliefs without reason.

But notice the opportunities. Consultants and facilitators have a theoretical and practical interest in such topics as

• the fit between individual and organisation

• the relationship between belief and behaviour

• change, stress, and coping

• problem solving and decision making

• learning.

In the decades ahead it seems that many challenges face us. We can anticipate a growing need for a good understanding of people and systems. Creativity and innovation will be in greater demand. A tolerance for ambiguity will be helpful. So will an ability to use trial and error when all else fails.

These are qualities which consultants and facilitators often already possess. We have a lot to contribute.


18

References

Aronson, Elliot (1972). The social animal. New York: Freeman.

Beer, Stafford (1975). Platform for change. New York: Wiley.

Burns, Tom, and Stalker, George M. (1961). The management of innovation. London: Tavistock.

Dunbar, Robin I.M. (1992). Neocortex size as a constraint on group size in primates. Journal of Human Evolution, 20, 469-493.

Emery, Fred E., and Emery, Merrilyn (1975). Participative design: work and community life. Canberra, ACT: Centre for Continuing Education, Australian National University.

Endenburg, Gerard; Lindenhovius, Jasper; and Bowden, Clive (1998). Sociocracy: the organization of decision-making. Delft, Holland: Enburon.

Hock, Dee W. (1999). The birth of chaordic organization: human resources or resourceful humans. Boca Raton, FL: KPMG International Executive Series, September 27.

Klein, Gary A. (1997). An overview of naturalistic decision making applications. In Zsambok, Caroline E., and Klein, Gary A., eds., Naturalistic decision making, 49-59. Mahwah, NJ: Lawrence Erlbaum Associates

Lawrence, Paul R., and Lorsch, Jay W. (1967). Organization and environment: managing differentiation and integration. Boston: Harvard University Press.

Lorenz, Edward (1972) Predictability: does the flap of a butterfly’s wings in Brazil set off a tornado in Texas? Paper presented at the December 1972 meeting of the American Association for the Advancement of Science, Washington DC.

Mintzberg, Henry (1994). The rise and fall of strategic planning: reconceiving roles for planning, plans, planners. New York: Free Press.

Naam, Ramez (2005). More than human: embracing the promise of biological enhancement. New York: Broadway Books.

Quiggin, John (1997). Estimating the benefits of Hilmer and related reforms. Australian Economic Review, 30, 256-272.


19

Rehm, Robert (1999). People in charge: creating self-managing workplaces. Glasgow: Hawthorn Press.

Reynolds, Craig (2001). Boids: background and update. On line. Retrieved 27 September 2006 from http://www.red3d.com/cwr/boids/.

Snowden, David (2004). Social complexity: tools, concepts and methods, 5 CD set. Melbourne: SOLA.

Snowden, David E., and Stanbridge, Peter (2004) The landscape of management: creating the context for understanding social complexity. Emergence: Complexity and Organization, 6, 140-148.

Taylor, Steve (2005) The fall: the evidence for a golden age, 6,000 years of insanity, and the dawning of a new era. Winchester, UK: O Books.

Weber, Max. (1921). Economy and society: an outline of interpretive sociology. Berkeley: University of California Press.

Weick, Karl E. and Sutcliffe, Kathleen M. (2001). Managing the unexpected: assuring high performance in an age of complexity. San Francisco: Jossey Bass.

Woodward, Joan (1965). Industrial organisation: theory and practice. Oxford: Oxford University Press.