The Challenge Network

   back   menu   next   

On decision-taking and its machinery

On decision-taking and its machinery

The relative advantage of the complex economies lies, increasingly, in their very complexity: in their ability to do things which less complex societies cannot. When access to human resource, savings and physical endowments converge across the world, then there can be no other grounds for sustainable differences in wealth. It is, therefore, important that complexity be harnessed, and that it be harnessed at speed: the wealthy economies need agile complexity management.

This essay considers how decision-taking can operate in a sharply more complex environment than is the case today, and points to some fairly radical conclusions. To get to these, however, we must think about how decisions get taken and what constitutes excellence. We regret that much of this discussion is theoretical, but it is hard to make progress without rethinking what is a fairly abstract area in its own terms.

Background considerations

Background considerations

Well over half of the output of the industrial economies now consists of patterns of order: of information, of the means of handling complexity and friction, of systems of organisation. The production of these intangibles relies upon an often equally-intangible infrastructure. Industries vary in the complexity of the inputs which they need. What is required to support commercial peasant agriculture is already quite complex, for markets need transport, transport need fuel, fuel needs refineries and oil wells. This is, however, almost negligible when compared to the support network that is needed to keep a centre of excellence in, for example, pharmaceuticals in operation. Much of the required infrastructure is intangible - legal structures and systems of regulation, for example - whilst some of it is grounded in the legitimacy which societies grant to industries to operate. Losing this legitimacy is a major threat, much as Germany crippled its nascent biotechnology industry a decade ago, and the US is currently discarding its lead in stem cell technology.

The market sector and the public interest are intensely and intricately intertwined, and decisions cannot be taken effectively - for the long term, delivering resilience - in either of these without reference to the other. No significant industry in the complex world operates without regulation at some level, and the quality of this may well define its profitability. Profit in the oil industry comes less from producing oil than the permission to produce, and from the tax deal struck with the state.

Complexity is managed through a fluid amalgam of market forces, through the actions of the state and from the workings of society and its history. In the less complex countries, the general machinery by which this is done is relatively comprehensible, however dark the inner workings may be of its individual processes. In contrast, the inner workings within the complex world are largely transparent, but the overall engines are so tightly interwoven, and so complex, as to defy analytical understanding. A recent British state report on management education ended with two page glossary of acronyms. One takes the machinery of such a society on trust, or drowns in acronym soup.

The three distinct spheres of the marketplace, the public interest and private lives change at different speeds, and in response to different stimuli. Commerce is driven by familiar and accelerating forces, and its internal agenda is relatively uncontentious. It goes ahead or it dies. Some few aspects of society move quickly, but most of the deep features of it alter relatively slowly when compared to commercial change. More specifically, the consensus view of how to think about 'our' society ('us') and my role in it ('me') is negotiated in a loose network, horizontally through age cadres, vertically through clusters set by income and education, and more generally via journalism, advertising and entertainment. Whether there is a problem to be solved - or how to think about new options or new insights - is something which is usually articulated by one or more affiliated interest groups. If the issue matters to the society as a whole, then a broad, consensus view is gradually negotiated across it, always grounded in existing ways of discussing related issues.

There is plainly scope to perform these tasks better - more quickly, in a more coherent manner - and an imperative to develop this. However, tactic views and intangible infrastructure cannot be simplified away without considerable loss. As we have noted, the pressure is on societies to cope with becoming more complex, not to streamline themselves or to set out to become more simple. Individual companies did, of course, simplify themselves during the 1990s, using 'core focus' in order to respond to increased external complexity. They were able to do this because a myriad of new companies were springing into existence, and able to accept outsourced activities as a part of their core activity. This is not, however, an option which is open to society as a whole; and whilst companies may have become individually conceptualy more simple, the economy as a whole became much more complex. In addition, each individual company became capable of handling much more complexity, but now within a a focused area of concern.

Society as a whole is becoming more complex. We have more options, and more confidence in exercising them. We expect more quality, less risk and fewer constraints. Connections between hitherto isolated domains of activity make every action ramify through the resulting system. To cope with this, we need to grow our related managerial capacities at the same rate or even faster. We need to manage dissent into a positive contribution to an outcome, not into conflict. Above all, we operate in societies which contain very many 'knowledge domains', pools of expertise and insight that view problems from many perspectives. "Good" options and choices come when all of this knowledge is constructively deployed on relevant problems. Commercial frameworks will oversee to some aspects of this, but they cannot provide all of the answers.

In summary: Complexity is innate to the industrial economies, and the capacity to manage complexity will define those groups, societies and industries which do well. The distinct domains of social, commercial and public interpretation of events will become more interconnected. How this will evolve is unknowable, but there is plainly much potential for misalignment and friction. The positive potential which exists for those groups who take a fruitful approach is, however, very great, and the remainder of this essay looks at the potential shape of what this might be.

Gaining understanding, finding options and making choices.

Gaining understanding, finding options and making choices.

Issues may present themselves, or they may be invented by interests who need anything from a marketing position to a figure of blame. This said, three are four distinct phases than can be recognised: issue definition, option generation, resource allocation and execution. Relatively trivial issues tend to follow a linear path through these phases: we have a problem and here is how to think about it; here is what we might do; is there anything else we would rather do with the money; and so let's do it. Complex issues follow a much less straightforward trajectory. The triangle, shown below, identifies some of the more common way stations and problems.

Additionally, it is much harder to manage towards solutions or amongst issues which lie a long way from areas that people are happy with, from traditional concerns which are well understood and which are addressed as a matter of habit.

The figure contrasts these dimensions of complexity and familiarity. Extremely complex, unfamiliar issues - shaded red - are addressed by general social mechanisms, usually indirectly and in a highly non-linear and contested manner. Less complex and familiar issues are handled by conventional, linear problem solving mechanisms, an area shades in blue. Between these, two wedges are shaded green and yellow. In the green segment, issues are debated and promoted by interest groups, either for or against the matter. In the yellow area, the primary agents are the state, which has to take general social concerns and make concrete policy proposals from these. (The exact relationship of these areas matters less than the immediate feel which most readers will have for the quality of debate in these segments: shapeless debate, wrestling for tractability, opinionated lobbying, day-to-day operational choice.)

Consider the three red arrows, symbolising the path of an issue from its inception to its routine management. Line A might be the decision of a company whether to upgrade a facility or not. Line B might debate technical issues about defence policy: to invest in heavy lift aircraft or fast sea freight. Line C might symbolise the debate on contraception, already discussed.

Line A may well involve huge expenditure, and Line C none at all; yet the former is conducted within a single frame of reference, and is tractable to efforts towards quality control , in a way that Line C is not. We know how to handle single-domain debate; and we still use techniques that would be immediately recognisable to the authors of the US Declaration of Independence when we attempt to bring quality into multi-domain discussion. It is not merely that the terms of reference vary - that what is acceptable as an answer to the question changes with each stage - but that the stages are not sequential. Rather, the entire process is usually conducted in parallel, or often in reverse, with a "tractable" decision being challenged by a media panic and social reaction, an NGO offensive or state intervention; or all of these at the same time.

The pathway of an issue can also be seen as crossing a space in which two important dimensions are in place. On the one, progress is limited by understanding - understanding the nature of the problem, understanding what to do about it - or limited by resource. (We have, of course, already seen this sequence, as the 'linear' decision-taking process. The other limitation is, of course, that of the general acceptability of the project. Sticking with the positive, the horizontal axis shows a range in which stakeholders are either broadly complaisant or demand to be convinced. (The colours - red, yellow, blue - map to the colours used on the preceding figure: issues dominated by social debate, technical assessment, operational excellence. We have omitted green for simplicity.)

The four quadrants have characteristic qualities. Where the public demand answers but where understanding in scant, then there are hard 'social' questions to be addressed. At the top left, where resources are limited and the public demand to be convinced, conventional politics plays itself out. The vertical red arrows show the repeated dips which politicians make into the pool of ill-defined areas of concern, and their attempts to create operational policy from what they catch.

In the top right, commerce operates without many resource limits and with the general acceptance of the public. Lower but still on the right, we find the space where understanding of the issue limits what can be done, but where the society neither knows nor cares about this. Here, only the enquiring mind enters, but from this quadrant tend to come innovation and new ideas, usually hapless in their lack of societal context.

Two red lines flow from this fountain of new ideas and insight. One, labeled A, shows the expected trajectory of a new possibility: it finds out how to undertake the details of what might be done, acquires resource, becomes a routine issue. This may be what happens to many uncontentious issues, but Line B shows what is much more likely to occur to ideas and innovations which have great impact. The new project is thrown into the area of poorly defined social concern. (In fact, this often happens after the project has climbed into the blue area on the figure. The project then spends a great deal of time in the red and yellow - regulatory - areas.)

The technological potential of the next 20 years will throw up a large number of projects that follow line B: biotechnology will give us life extension, the ability to manipulate mind states and mental ability, the capacity to alter the inheritance of a child as well as the ability to discriminate amongst people as to who has the capacity to learn certain things, or has the predisposition to crime, violence or sociopathy. Information technology will spell the death or privacy and the need for security in spheres where we felt safe; will change the labour market through artificial intelligence and telepresence; will both enable in astounding ways but also deny the liberty to cut corners in others. Economies which prove able to manage the onset of this potential will do better than those who find it denied to them through social rejection.

It is relatively easy to say what a 'good' process or decision look like. From what we have learned so far, ideas which are judged to have been good are likely to be:

Poor solutions are all too common. Later, we shall look at conflict and conflict resolution. However, the diagram below points to one important pathology. A solution works or it does not (vertical axis) and is popular or not (horizontal axis.) Many of the matters which we have already discussed - and much policy - fits into the 'bitter pill' segment at the top right: painful issues which need to become accepted as a part of the tool kit. The blue arrow traces the required flow, as monetary management has processes from painful wild idea to central dogma over the past generation.

The two white arrows trace pathology. On the right, the bitter pill is downgraded by propaganda, so that something which in fact works is seen to be useless. Islamic fundamentalists have had polio vaccination banned in Northern Nigeria - leading to epidemics - because they have persuaded local people that (a) it is useless and (b) even if it worked, it was an example of 'Westoxication' and best banned anyway.

The lower left quadrant is highly populated with useless measures, from quaint superstitions to equally quaint but costly corporate rain dances. In the absence of effective measures, pour a little from each glass of wine onto the floor, for the gods. A highly popular measure is to take such a procedure and make it policy, often criminalizing victims in the process. There is no need to cite examples.

A further force which can weaken solutions is the following. The figure that is shown below repeats the axes which were used earlier: the complexity and the novelty of an issue. The chart shows a series of diagonal lines, representing equally difficult combinations of these. One can have a very complex familiar problem, or a simple but novel one to solve and expend the same amount of effort on both.

Consider an issue that is to be taken up by one such diagonal line - that is to say, its game is to be raised. This is symbolised as the red dot. Two trajectories move up to the next level of difficulty (although, of course, there are an infinity of such lines.) However, Line B is the lowest line which would not demand a reduction in the complexity which the project could handle. As we have seen, the capacity to handle complexity distinguishes industries and economies with high and low incomes.

Line A, by contrast, is one of a family of trajectories which hugely increase the amount of complexity that can be handled. This is done by focusing on the familiar - the words of the 1990s, by "sticking to one's knitting." The upshot is a significantly lessened capacity to handle new things - specifically, to innovate and often, to operate in new markets. Indeed, the flavour of the 1990s was to break activities into modules, and to farm out to third parties any module which was not "core". This undoubtedly allowed the complexity associated with scale, but arguably at the cost of insight and anything but the most obvious of innovation.

The figure above repeats these axes, but separates the public and market sector. The angle of the 'lines of equal difficulty' differ.

The public sector is, for all of the reasons which we have developed, almost always faced with issues which are more complicated that those which confront individual commercial organisations. Often, therefore, a small amount of potential innovation has to be set against considerable complexity to be managed. The gradient in the 'pay off' chart is shown above, and the resulting trajectory suggests and even more pronounced tendency for state enterprises to focus on complexity management at the price of innovation.

This said, the consequence of business - as first mover - stripping itself to the core has been to dump much of the complexity that it once managed onto the state, where the regulatory burden has grown sharply. This places even more weight on movements up and to the left of the figure; and whatever is said in policy circles about innovation, only those changes which sharply reduce the burden of complexity to be managed are remotely attractive. An example is the outsourcing of activity back to the private sector, by contract or by privatisation, but under focused and semi-detached regulatory control.

In summary: we have learned that policy choices move through discrete stages, in which how they are thought about, who is concerned with them and what criteria are applied to them vary with the type of issue and the state of maturity of it. We have noted that there are some strong and natural forces which set out to blow the more contentious of these processes off line. Third, it is evident that decision-takers prefer to be able to focus on what they know well, and to do it better, rather than to take on new things.

Disputes and debates: getting to clarity.

Disputes and debates: getting to clarity.

The introduction noted that complex problems are solved in non-linear ways. For clarity, however, let us pretend that there are clear stages in a debate. In such an artificial scheme, there are three phases in which an idea is developed. First, individuals and groups who are interested create a model of the 'problem space' - of how to think about the issue and the things which affect it. Second, this model is used by them - or transferred to others - in order to understand the implications of it. Such implications include the risk and reward associated with various kinds of response, with the trade-offs and resource needs that these present, and the relative attraction of a given option when compared to other uses of the same resources.

Third, the chosen option - or different options, chosen by different groups (companies, countries, individuals) - is acted upon, and these actions impinge on society, the economy and so forth. A working through of these impacts, lessons and examples tends to occur in the way that these groups think about their world, and this often provokes new issues, options and actions. Consider how Eighteenth century Europe came to terms with cheap distilled spirits, or how contemporary society handle the Internet and mobile telephones to see the ramifications of this.

Solutions may be agreed, unclear and so 'difficult' or may be more or less clear but actively opposed. Those situations of the middle ground, in which we face a 'difficult choice', but for which there is no political opposition - also fit within the three phases of issue development.

First, there are difficulties when the modelling of the issue is itself unclear - when there is inflation, for example, but we do not know what causes it.

Second, we may understand the problem but the outcome may be uncertain: should I invest in that company, or not? Equally, the data may be scant in that we cannot make a choice, even though we understand this issue, because we do not have enough information. "I must get to York by sunset, but where is a signpost?"

Third are those concerns in which all is analytically clear, but for which we do not truly know what outcome we want to achieve - or where the means to be used towards a desired end are not as we would wish.

These distinct forms difficult present us with distinct kinds of balancing act. We can never act in complete certainty and usually must take a choice in near-ignorance. We are seldom faced with a choice about which we feel unambiguously positive, both in terms of means and ends. This said, few important issues are unopposed, whether their analysis, evidence and value structure is clear or not.

Disputes also arise at each of the three levels. Those which are concerned with fundamental interpretation have, throughout history, led to the most profound antagonism. Those which are about option selection and generation are the basic stuff of politics. Once the rules are established, however, there are endless granular disputes about fact and balance, default and dereliction, and this is the heart of what law and policing addresses.

On the left of the figure, we have a simple matrix which is closely related to one which we have already seen. The vertical axis notes that we can be limited by our understanding of a situation (Phase 1, above) or, through a continuum, by our ability to carry out the prescriptions which eventual understanding may offer to us. However, any one of these steps can be opposed or generally supported, as shown by the horizontal axis.

The right of the figure uses this same set of axes to define a space, and sets out the kinds of consensus and conflict that we find. Disagreement as to fundamental analysis leads to stratospheric intellectual battles, for example, or perhaps to anxiety and mutual distrust. By contrast, consensus creates the tools for practical action, symbolised by the blue arrows running up and down the right vertical part of this space. Action taken under profound disagreement is always likely to generate conflict, and the equivalent space on the left of the chart is shown with red arrows.

In the middle of the figure looms a mountain. It is placed there to indicate that it is impossible to go from conflict and warfare at the level of operations to peace and harmony without ascending into the heights of analysis. One has to resolve the issue at the level of conceptual analysis - Phase One, in our simplified scheme - and no amount of compromise or pragmatism will serve. If one view prevails through propaganda, force of arms or economic weight, then it is left the sole contender. If the issue becomes less contentious, and people simply stop worrying about it, then a synthesis may emerge, although skewed to some other issues. However, so long as the matter remains heated and the issue one of interpretation, then nothing will be settled until the parties agree on a way of looking which works for them.

The diagonal lines suggest the impact of a new paradigm: a way of seeing that highlights self-interest in one of the groups, a perspective which allows for synthesis between previously rival ways of thinking. New insight can be as destructive of consensus as it is the source of healing. False models - that this or that religious group caused a plague or earthquake - is as potent in creating or solving conflict as analysis which eventually supported by the full weight of evidence: that mosquitoes carry malaria, not marsh mists. New paradigms need neither to be right nor to be socially positive; merely new.

The 'real world' is more complex than our three stage model, even when modulated in the extremely complex ways which we have just explored. This is because few issues run in the manner shown, and there are complex layers of feedback that lie across each stage in the process. We do not come to problems completely fresh. We bring our values, prejudices, special interests and blind spots. We also bring an immense amount of learned insight, particularly when we do this as a group. The greatest strength of a democracy is not its ethical basis but the degree to which broad sweeps of insight are brought to bear on any important issue that may be raised.

Insight and experience are much more than a codification of grand models of how things work. Rather, at all conceivable levels of aggregation, seen from all manner of perspective, what works and what does not, the 'why' and the 'how' of life are pooled when groups think about problems. Frequently, the matter is not articulated so much as filtered through personal intuition for how it 'feels', much as one learns as a child that certain actions and postures are best avoided when riding a bicycle.

Such insight has been filtered through many challenges and mutations to arrive at its currently most useful form, and the continued competition amongst ideas and options ensures that the evolution of language proceeds quickly and, with luck, usefully. Insight which is shared out amongst many people acquires a number of important properties.

One of the more useful insights from the theory of dynamical systems is that those which have the most complex properties are poised on the edge of dissolving into chaos. If one imagines two axes along which a system may vary - for example, the number of rabbits in an ecology, and the number of foxes that prey on them - then these independent axes, when set at right angles to each other, together will make a square area or 'space'. Any combination of these two properties is represented by a point in this space: a few foxes, lots of rabbits. If we let the system evolve - that is to say, the foxes breed, the rabbits run out of grass - then the point may stay where it is, or it may move around. The trace of the movement of a point over time is called its trajectory, and the pace through which it is moving over time is called its 'phase space'.

If one considers all possible points and their evolution, then one gets a remarkable insight into how this system behaves as a whole. The figure which is shown below is a sketch of such a system. Each trajectory shows how a given start position will develop over time. Remarkably, many such systems repeat themselves, often in less smooth ways than are shown in the figure, perhaps hopping between two fixed locations. Others form smooth orbits around a centre, or home in on such a centre. These locations are known as 'attractors', shown here as solid circles. Areas never entered - 'repellors' - are shown as an empty circle.

Life in a fixed orbit or an attractor is very dull, as there is no possibility of change or increased complexity. Societies which are locked into frozen stasis in this way are apparently stable, but often very brittle. A small perturbation can 'break' them, sending them into a situation with which they cannot cope because they have not evolved the toolkit through which to do so. They are fitted operate only at a lower level of complexity than their long term dynamics will permit.

Note, too, that the upper attractor has a swirl of trajectories that go around it, the outer ones of which can break away to fall into the lower attractor. This unstable situation is also not a happy one, as it is disruptive and, in complex systems with many attractors, chaotic. Stable systems that can adapt themselves to externally-applied changes - more rain, a drought - are those which are neither prone to chaos nor locked into a repetitive cycle.

Most ecologies contain huge amounts of redundant but latent response to change. A teaspoon of fertile soil will contain around a billion organisms, and between 500,000 and two million species of micro-organism. Studies on soil DNA show that most of these species are very, very rare and that the ecology is dominated by a few species. However, if the situation changes - more water, different food stuffs, then the rare become common and the hitherto commonplace organisms become rare. Resilience comes from alternative responses to changing conditions.

Societies support parallel and dissonant models much as ecologies support different organisms. Resilient societies actively develop this diversity, and seek useful approaches when conditions change. There are two levels at which this is done. First, as discussed above, people and groups are guided by models of appropriate behaviour: when risk should be seized and when safety adhered to, when assertion is appropriate and when to mollify the powerful. Second, of course, there are many rival formal models, varying ways of looking at the world that are in play. Societies which suppress these - as with totalitarian regimes and theocracies, for example - have no nuance to deploy when confronted with the unfamiliar. Members of the society look to precedent or to pronouncement from the leadership when they need to take a choice. The leadership itself can draw on only a tiny fraction of the available knowledge, filters this through orthodoxy and is anyway concerned to produce a response which is ideologically correct rather than operationally useful. Where stupid ideas collide with demands for concrete results, expediency conquers morality and exploitation follows.

Dissonance, transparency of information and plurality of access are plainly good things to have but, like oxygen or water, too much applied in the wrong way is nevertheless fatal. Once again, the issue is one of how much complexity a given system can process. We have already discussed the various characters of conflict. Ultimately, a society which is in disagreement will drift around in its phase space until a trajectory is found that removes it from conflict; or it will settle into an orbit which repeats, either suppressing dissent or oscillating between expressions of it.

Societies which are prone to self-examination, to listening to differing perspectives, to trying experiments is likely to find these helpful trajectories sooner than those which do not. Perhaps more to the point, however, such societies will have many prepared minds which chance discoveries may favour; and totalitarian ones will not.

This capacity to know a good thing when you see it characterises adaptive social systems. For example, it is likely that an industry made up of many medium-sized companies will pick up a good idea faster than a monolith of similar overall size. However, very small businesses will not have the analytical power to identify anything but the most self-evident of ideas and will probably lag behind the monolith. Innovation in microelectronics is said to peak in the 120-150 person company.

The speed with which winning ideas are identified by scrutinisers- companies, rival groups within a society, policy processes - varies, therefore, with familiarity, complexity and transparency of the problem in question. There is also an important element which is due to the articulacy and insight of the groups which are involved. An issue which is unfamiliar, presented to people who have never thought about it before, and presented in the absence of clear facts will, self-evidently, take much longer to be processed by these people than its converse. However, many issues in the modern world are thrust onto stakeholders in just this way.

Simple, slowly changing societies had the luxury of decoupling all of these stages from each other in time. In addition, they were able to segregate the social roles in which innovation took place. New ideas were, therefore, tried out in the laboratory setting of aristocratic circles, amongst scholars, in the monasteries or amongst the soldiery. Successful ideas percolated slowly out of these confines.

Modern societies are not structured in this way, for there are very many sources of new ideas, interconnected in equally complex ways. There are intermediaries who re-present ideas in new frameworks. We have already discussed how societies offer individuals template-identities. These once tended to circumscribe the interests that those individuals might have or the ideas to which they might subscribe. A bank manager of fifty years ago would have vested his behaviour in his occupation: how he dressed and lived his private life, from hobbies to transport, and how he behaved to others in his community. Someone in such a role today might do all manner of things - from rock climbing to rock music - and would see their occupation as only one facet of who they are.

One consequence of this is that such people have multiple sets of values installed, to be used when appropriate. Rock climbing and client management require distinct approached to risk and spontaneity, for example. However, such value sets are not loaded and unloaded like computer packages, as need dictates, but rather vie for dominance. Few educated people in the complex economies now have a single set of values, but rather operate under a flexible set of interactions and pay-offs, a system which has set limits, to be sure, but no certain centre ground.

The implications of this are, of course, profound. The great engine of society runs on implicit assumptions which change gradually, and as they change, alter how the systems managers act. As these models become increasingly evidence-based, so they circumscribe the possibilities open to managers. The options which they present are extremely powerful, but they span a narrow range. A generation ago, it was possible to pose a Marxist or a Capitalist model of economic conduct as proper for an economy. Whatever the label applied today, the issue of 'how one manages inflation' is well-understood, as well proven and any other domain of knowledge and full of powerful tools. However, central banks have very little freedom within these models. Asking a similar question, one cosmologist asked whether God have any choice whatever in how the universe was constructed, or was what we see a necessary outcome?

In an elite cadre, therefore, Absolute Truth locks the system into a fixed orbit. Debate is conducted in terms of the official model. Rapid policy responses are made to the fast changing environment. Complexity is shunned, as the system is itself enough to concern its managers. To the population at large, however, a black box has appeared with systems guardians who speak the arcane language of priesthood. Intermediaries exist - journalists, NGOs and other interest groups - who make it their business to interpret (and usually criticise, rather than applaud) the priesthood. So long as everything works, this is acceptable to all parties. When it does not work, then the public becomes anxious about the guardians of its arcane.

Summary: A society which is good at solving problems is one that has a good, shared understanding of 'how things work' but which is not locked into a frozen assertion of this. It is good at managing multiple viewpoints, where this management prevents loud voices drowning debate. Conflict management is addressed where necessary by reaching to the fundamentals where dissent is grounded and finding new ways of speaking about the problem area. Societies which are good at solving problems allocate the management of issues where the consensus is strong to delegated authority, but maintain mechanisms to avoid these becoming elite 'priesthoods'. Excellence demands that such matters are sufficiently networked between the domains of expertise and the general public such that trust in them is maintained. As we shall see, however, this is a difficult task.

Trust and party politics.

Trust and party politics.

Experimental economics is a burgeoning discipline that attempts to measure issues on which economic theory has hitherto provided only reasonable assertion. (See also here.) These papers describe two important findings.

First, in both humans and other animals, considerable weight is placed on maintaining tacit institutions - trust, fair play - to the degree that individuals will takes steps that cause them individual harm in order to protect social goods of this sort.

Second, trust is understandable as a contract between two parties so as to reduce transaction costs. That is, I trust you so that I do not have to police every step of any relationship between us. Plainly, if I can collect fruit while you see off thieves, then we both do better than if we both try to collect whilst nervously peering over our shoulder. We can specialise when we trust, and specialisation is the prime engine of economic activity and growth.

Experimental economics has developed trials which measure the concomitants of, for example, trust. The papers referenced above explore the details of this. In brief, however, we extend trust to those who have obvious motives to cooperate with us - as with the division of labour, above - whose reputation is good and whose behaviour we can at least partly validate. We also extend trust to those down on whom we can call sanctions when we find that they are not trustworthy. Excepting the last issue retribution, all of these issues are concerned with information and transparency.

We are strongly motivated to seek the trust of others. Reputation provides those who are held in good opinion with many advantages, such as preferential treatment. Remarkably, this even holds with computer programs. It is possible to design algorithms which trade with each other with or without cheating, and which 'pass their genes' on to their progeny in ways which reflect their success as a trader. Each algorithms has the equivalent of reputation, which governs how other programs treat it in the absence of other information, and which is created from their joint experience. Such programs 'breed' honest dealing, and successful algorithms generally give fair measure, with cheating at the margin.

Trust in the public domain is, therefore, predicated on these same features.

"Trust" is not the same thing as "liking", however: I can trust a politicians whose views I detest, and fail to trust one with whom I am in apparent agreement. The concept of trust can be extended to any coherent agent - individual, institution, alliance - but cannot be extended to anything which lacks the unit of motivation and responsibility which these imply. If the entity is not perceived as being coherent, or if it is perceived to be likely to evaporate when things become difficult, then it cannot be the recipient of trust. (One should emphasise the word 'perceive', for trust is about perception.) Brand - presentation, what sociologists call the "narrative" that the organisation propagates about itself - is extremely important to any entity which is to be extended trust, and thus to be able to operate in the public arena.

The unity of policy making in most democracies is the political party, plus its affiliated think tanks, lobbying organisations and other supporter groups. Parties operate in a world in which many pawns are pinned. They have a large supporter groups which is with them for reasons of history and habit, which they must not alienate. Real policy options are tightly bracketed by the issues of best practice and expertise to which we referred above. There are policies on, for example, monetary or fiscal policy which are simply untenable, given what we know about economics.

Additionally, however, policies exist in distinct arenas: one's views on defence have little to say to one's position on, for example, education. Attempts to find a coherent political philosophy which cut across all such boundaries simply do not work in the modern world. The voter is at liberty to like the defence policy of Party A and the education proposals of Party B with no loss of intellectual coherence. However, parties that are to win must focus on specific interests, which usually means focusing on specific demographic or attitude clusters. The once-overwhelming polarities of rich and poor, traditionalist and liberal are now fragmented and diluted so as to serve no obvious purpose. Indeed, the poor-elderly-traditionalist versus money earning-young-future oriented category seems likely to (weakly) supplant the left and right of the past hundred years of Western politics. However, few extant parties will be able to make the necessary migration, as the risk of shedding their former supporters is too great.

Political parties may, indeed, be a part of the problem rather than a key to the solution of how the West is to best manage its affairs in the face of increasing complexity. They have lost trust, and lost the underlying concomitants of trust. They are extremely poor as policy engines: ideas are created to attract electorates to a brand, are put together in ways that can only be called amateur and are never challenged in the way that we have seen to be appropriate for good ideas to emerge. Above all, the decadal cycles of political power - novelty, maturity, exhaustion - mean that policy renewal is always behind the times, and either struggling for relevance or swamped by the demands of power. Once in power, the tendency is for executive to shed with some speed the party baggage that put them their, and to react to the existing, expert-driven fragmented issues which we have discussed.

Parties exist because political power is about winning specific races, and collectives are better at managing this - getting votes, concentrating resources - than are confederations of individuals. They are the Darwinian solution to the political arms race. There is no doubt that they will continue to exist, not least as they would have to legislate themselves out of existence. They are, in fact, quite good at delivering human resource, organisation and accessible communications. What they are bad at is policy. Parties which contended over issues of competence and fitness for office - irrespective of what they were supposed to deliver - would probably be more honest about motivation, and certainly more trusted, than the current mixture of motives.

The issue is, therefore, the way in which the complex economies are to renew their policy engines. This essay has looked at how projects that end in decision are birthed, given substance and executed; at how dissent and disagreement can be harnessed in a constructive way, and at how disagreement is to be resolved. We have noted a range of pathologies and problems as we did this. One clear note that rings from all of this complexity is the need to manage matters in their own terms: that different stages of problems need distinct handling, that differing constituencies have different needs, that dissent is usefully brought on board through some routes, and exacerbated and made obstructive through others. In short, the policy process is much more complex than contemporary democracies allows. It seems at least reasonable to ask whether a separation of powers would not be advantageous.

In Summary: the primitive machinery of elective democratic politics has not kept up with the need for sophistication in policy creation. Brand positioning for political parties - and democratic selection processes amongst these - should not be confused with the acute need for better insight, more transparent and rational portfolio management and the fusion of constructive criticism, experience and technical insight. What politicians are tasked to do and how they set about doing it are completely distinct things.

Towards better decision-taking processes.

Towards better decision-taking processes.

The organisation of the political framework is relatively primitive when compared to commercial organisations, which also have the luxury of (usually) operating under internal consensus. Nevertheless, there are telling lessons to be learned from what we have just seen. Every modern organisation uses internal specialisation in order to run its affairs. The public sector and market organisations have extremely elaborate divisional structures, reporting hierarchies and the like. A typical large organisation will need to be analyses along five or six dimensions (line-function, regional-global, divisional, reporting hierarchy and so forth.) What could possibly be added that would lead to better decision-taking?

We suggest that the answer lies along a different dimension, one which is normally folded into the standard structure. This dimension embodies much of what we have discussed about the nature of decision-taking, and very little about operations, results and the like. Above, we reproduce a diagram used earlier, but with some embellishments. Readers will recall that this contrasted the limits set upon a project by it social acceptance, on the one hand and, on the other, by its emergence from limits set by understanding to those of limits set by the requirements of its implementation. The colours referred to earlier analysis|: red was an area of predominantly interpersonal communication, yellow that of technical analysis and assessment, blue that of routine operations.

We have added two features to this figure. First, there is now a dotted area to which the operational structures of commerce are comfortably suited, and within which its management is generally happy to operate. Parts of organisations which are forced to operate beyond these limits (renewal, external relations, thinking forward) are generally poor relations with limited connections to the main engines of activity. The political aspects of the public sector are, despite the rhetoric, set up for exactly the same tasks in the same area, although individuals within these organisations are often happy to operate in the red area of the figure; and think tanks and others in the yellow zone. This said, these capabilities are as little linked into the main structure of political power as are their peers in the commercial sphere. It is not that these zones of the chart are not populated, and often occupied by extremely capable organisations. Rather, there is a weakness of process, of legitimacy, of fundamental connectedness in this area of endeavour.

This brings us to the second addition to the figure: the diagonal arrow, shown in red. Here is the 'missing organisational dimension' to which we referred above. In commerce and in public life, flows up and to the right of the diagram are supposed "just to happen". Thoughts have been given to bringing in structure, but two standard objections to this are almost always raised. First, this area is thought naturally to defy attempts to give it structure. To do so would kill spontaneity, and bureaucratise what should be loosely connected. Second, even if this were possible, then how should it be tasked? Would senior managers - presumably? - not have to know the very answers which they sought in order to set targets? And so nothing is done

What we see in these objection is, of course, a baulking at the first stage in the process of decision taking, with which we began. First, one needs model of the problem space. Then one finds some options which match whatever it is that one wants to achieve. Then one filters these against other claims on resources, and executes those which pass through this sieve. This flow is handled, ad hoc and with many false starts, in any complex economy and every viable company on the planet. It is generally handled in ways which could be improved. So how is any organisation - or country, or subset of the state - to better organise the ways in which it arrives at 'a model of the problem space'?

This is not the place for detailed prescriptions. However, some very straightforward elements emerge from the simple contemplation of the new dimension of organisation, as discussed above. Here are some fairly obvious points:

If we extend these principles to either the governance of a large organisation or to politics, we see much the same result. One needs two chambers - boards - which act as mutual modulators and scrutinisers. It is possible to see a role for a third agency, to which we turn in a moment.

The first of these agencies - boards, chambers - is concerned with directly operational matters, with results, with performance and delivery, and with immediate stakeholder perceptions and concerns. In current British terms, it is the executive and parliament; or in commerce, comprises the operating board and line management. By contrast, the second has no direct analogy. It is concerned with the issues of integration which have just been discussed: with renewal, with foresight, with the ill-defined and the poorly communicated, with communications and knowledge, with uncertainty and insight in the broader community of interest. This agency has its reporting structures and resource base and these have a direct process relationship with those of the other board. More pertinently, however, each agency has power over the proposals coming from - ratified by - the other, and interventional powers of at least critique. It can summon a representatives to explain decisions which they have taken.

This structure may of itself be enough. That is, enough to fulfil the key questions which we have posed for ourselves: how can we manage increased complexity with better insight, more speed and with greater flexibility? How can we harness constructive dissent and pluralism of insight and knowledge, whilst avoiding paralysis brought about destructive or predatory dissent?

It is, however, possible to envision a third agent, aimed to stand somewhat back from the day-to-day running of affairs, much as monarchy is supposed to admonish and advise. The role is partly played by the Supreme Court in the USA, and - tenuously - by the European Court of Justice. This third agency stands over and above the two so far described, and is concerned with the overall balances that are being struck. In the most enlightened company, shareholder value is seen as coming from both acts of commission and acts of avoidance - with excellent performance and surprising renewal, from risks avoided and harm not caused. This third board enforces the implicit values of the society, and interprets the intent of past legislation and, where relevant, the popular will. This role is currently assigned to the integrity of the elected chambers, and supposedly enforced by the elective process; whilst actually managed chiefly by media critique. Might it be better to formalise the role in some way, so separating powers? In companies, this is the notional role of the non-executive director and of supervisory boards. Once again, a formalisation of the role might prove helpful.

In summary: The extremely complex nature of decision-taking means that powerful people have focused their attention on the more tractable parts, and where their focus is drawn, other people of power are drawn. It is reasonable to imagine that re-weighting the structures of power so that more attention is given to the less tractable and rewarding aspects of decision-taking could be helpful. It would help companies and countries manage their way through complexity, avoiding dispute where possible, actively forging consensus and deploying the best possible opinion and knowledge in pursuit of options for renewal.

Feedback on the issue of taking choice.

Feedback on the issue of taking choice.

This text summarises various items of feedback from the first note that was published. Thank you: please keep comments flowing. One correspondent sent me a larger note, which is going to be posted separately.

All agreed that the notion of a decision having a life cycle - going through a larval, pupal and adult stage, so to speak - was correct. Broadly, there was a stage of getting the tools of thought, a stage of assessment and a stage concerned with roll out under fine tuning. Several writers said that the emphasise in the literature on decision-taking was all about these last two stages - portfolio ranking, risk assessment and pricing and so forth; and on project management, post-project appraisal and the like.

Four issues were raised as needing attention:

By contrast, the blank canvas was concerned with the first step, and it was here that things had to change to meet the needs of a more complex and vocal society.

Recognising a false model

Recognising a false model

There was a great deal said about incorrect models. The issue was how we were to recognise the difference between the situations in which our basic way of thinking was incorrect, and those when our information was simply too poor for a good choice to be made. In addition, our choices are as much rooted in heuristics - rules of thumb, unspoken assumptions, values - as they are 'thinking models'. How are we to give proper balance to each?

These are important questions, but the fundamental issue was thought to be this: how do we recognise a bad way of tackling an issue, and move on? Plainly, the answer is through the carrot of clear answers and options, and the stick of failure. Whilst this is obvious in simple issues, however, it is far from clear in more complicated ones.

Ways of thinking which are socially-embedded are particularly hard to change. We prefer to blame the data or the messenger rather than to change our collective minds. We blame leadership, for example, rather than the principle under which these individuals are operating. Turnover in the boards of companies with an obsolete business idea tends to be high, even when they articulate the problem and the need for change.

This is particularly true when we have to abandon a valuable way of thinking for a void. It is much easier to drop, for example, communism as our guiding principle when there is an alternative glittering in front of us. The figure which showed the balance between complexity and novelty which an organisation or society could manage was thought to be highly pertinent. If we are offered a rival complexity management system we will be happy than is we are abandoned with none.

A development of this figure is given below. The same axes - of complexity, novelty - are broken by two arbitrary lines. The vertical line separates those things for which we have a familiar regime and those which we do not know how to handle. The horizontal line distinguishes areas where the issue is plain and obvious, where one cannot evade the fact that there is an issue to handle. Above the line, however, there are ten ways of looking at every issue and it is at least arguable that there is no problem, that the person debating it is seeing it from the wrong angle, with bias or in other ways which allow denial and misdirection.

The four spaces are plainly distinct, particularly when viewed from the perspective of how we come to abandon a false model. Political activists operate in the top left, bidding for legitimacy by trying to show that the "routine" box is in some way invalid - socially unjust, religiously inappropriate and so forth. The intellectuals try to show that the routine is wrong for other reasons: that things would work better if it were to be changed, through monetary management, for example. For reasons discussed above and in the original paper, the 'routine' quadrant finds it difficult to declare itself invalid, but it can withdraw trust and legitimacy from either of the other two and feel uneasy with itself.

Additionally, each of the other two quadrants can go to war with each other - a contest of brief duration, as the rationalists always lose in the short term and may win in the longer run, by osmosis, if their ideas prove to have legs - and they can fight internally. That is how most of political life is lived, and - says one correspondent - virtually all life in academia. These ideological, theoretical squabbles generate intense heat but usually have little impact on daily reality, save to weaken trust. When they do sweep up a society, however, they change it as no other force.

Summary

Summary

Comments were that the long, theoretical note had useful things to say, but that they needed to be dug out and made more clear. The emphasis o comment was on the ragged way in which decisions are usually made, and how this could be improved. People were struck by the "social" aspect of decision taking, and by the time, translation and resource which this needed is it was to be done well. If there was one view, it was that the knowledge economy needed very much more effort put into managing and understanding these issues than is done to date. A minority held strongly and that successful countries and companies would be those which were able to cope with complexity with a smile. A broader view was that if its implementation was not stand-alone a recipe for success, its absence was certainly a stand-alone cause of failure.

Further feedback on the issue of taking choice.

Further feedback on the issue of taking choice.

I have read the paper on decision taking. It was not what I had expected, which had thought was going to be a rehash of issues about risk and evidence based decision taking. In fact, I thought it very original and helpful. However, I would like to add some thoughts, which you can publish if you want. I have not got permission from my masters to put my name to this, so please do so as an unattributed article. My apologies for this, but it would take weeks to get a release.

I want to talk about the cost of choice, and why too much choice can sometimes be a bad thing. (Interested readers are referred to B Schwartz Why More Is Less, Harper Collins 2005.) People who have run the gauntlet of ordering Starbucks coffee will know what both he and I mean by this.

Two situations where over-much choice presents itself are:

Poorly educated people have to trust more than well educated, insightful or intelligent ones. So that group in society are the least trusting, and get cheated the most. (For other reasons as well.) As things get more complicated and expert, they are going to be in an increasingly bad position. Also so will the poor countries, which can't afford the best advice, or have no useful internal forces to harness.

Now, these two situations can develop 'sick' situations, what you called pathologies. I like the word. There are three pathologies in the big, grand area where important things need delegation if we are to get to good solutions.

The key is that (1) the interface has to be intuitive and easy to use, and not embarrassing - so not a service outlet where connoisseurs might be listening, with curled lips. And (2) the interface has to make learning about deeper levels easy too - a person can choose basic black, gray or red, and then dive down into twenty kinds of red, ten kinds of finish if that is what they want. If not, the result will be perfectly satisfactory. If so, it will be unique to them. Well, fine: but this is a new approach to self service that could be applicable right across the board. More self-education than following a decision tree to the twig that best suits you.

I am not sure how this contributes to the 'scenario' debate, but it certainly reinforces my view that decision-taking is very complicated. If we are going to do it well, we need to know which 'space' we are in, because one kind of approach that works in one area won't work in another. Stress conselling and facilitation may help in a peace negotiation, but does not help when you are trading securities. I don't think we have the tool kit that works identified, and I don't think we know when to use a plumber's tools and when to use a bicycle repair kit.

 

  To Top