The Uses of Pessimism

From many years of teaching social policy analysis to graduate social work students, I have come to expect that what students most want to learn is how to advocate for policy changes—changes that they already ‘know’ are needed. If they are concerned about a – any – social problem, their first instinct for addressing it is to say, “There ought to be a law.” The law, whatever it is, involves more state intervention in the lives of families, neighborhoods or other “mediating structures” that stand between individual and state. The idea is that the state should intervene to protect individuals against those structures and networks, from what Edmund Burke called the “little platoons.” In this vein, rights are understood, not in their original sense as the just claims of individuals to freedom from the state, whether in religion, security against arbitrary state actions against person or property, but rather as claims on the state.

Most mistakes in policy analysis, as in philosophy, result from moving too quickly. The first task of a teacher in either field is to slow students down and help them to be more aware, if not critical, of their own assumptions. In this respect, Bardach’s A Practical Guide for Policy Analysis warns about the “optimism problem.” It is an ethical problem, he argues, and a professional obligation to consider what could happen, in particular what could go wrong, if people were actually to follow the analyst’s advice. The costs of over-optimism in the lives of real people, as well as the opportunity costs of ill-used resources, are almost never borne by the analyst who gives the advice. All “solutions” have costs or trade-offs. Or as Sowell puts it: “There are no solutions. There are only trade-offs.”

In a cautionary article in the Harvard Business Review, Bazerman and Chugh (2006) argue: “The ‘bounded awareness’ phenomenon causes people to ignore critical information when making decisions. Learning to expand the limits of your awareness before you make an important choice will save you from asking ‘How did I miss that?’ after the fact.” They discuss various ways in which the blinders we wear keep us from seeing what could go wrong, even when the information is right in front of us or readily available. They discuss the Vioxx case, the Challenger disaster, the New Coke debacle, the Iraq war decision, among other examples. (See also the discussion by psychologists Dan Lovallo and Daniel Kahneman, "Delusions of Success: How Optimism Undermines Executives' Decisions" (Harvard Business Review, July 2003).

Unscrupulous Optimism
Roger Scruton, in his recent meditation on the uses of pessimism, writes as a philosopher and social commentator rather than as a psychologist or policy advisor to governments or corporations. His concern is also with the dangers of false hope (his subtitle) and the particular fallacies that make such “unscrupulous optimism”—the term he takes from Schopenhauer to distinguish it from a scrupulous and constrained optimism--so powerful and impervious to reason. The fallacies he considers include the Best Case Fallacy (i.e., the failure to consider scrupulously worst case scenarios), the Planning Fallacy, the Utopian Fallacy, Zero-Sum Fallacy (I fail because you succeed), Moving Spirit Fallacy (i.e., the Zeitgeist not as a way for historians to periodize past developments, but as dictating present and future directions), and the Aggregation Fallacy (treating freedom and equality as able to be added to each other rather than as contradictory or trade-offs).

In the abstract, and emptied of content, these are useful cautions that no-one sensibly could dismiss out of hand. But Scruton has a more important and more polemical purpose. He aims to show how these fallacies pervade a larger social and political vision that has been ascendant since the Enlightenment and especially the deadly triumph of “Reason” in the French Revolution. It is an unscrupulous optimism that sweeps away the collective problem-solving of generations codified through customs, traditions, and laws built from the bottom up, like English and American common law or Swiss political arrangements, with the will of the radical and enlightened few. The utopian or planning elite sweep all previous traditions and practices aside, along with the wishes of ordinary people, who have to be led to a higher level of wisdom by the progressive, forward-looking vanguard.

Scruton’s critique bears a strong family resemblance to that of Thomas Sowell (A Conflict of Visions), who sees the underlying divide between political and economic camps as the result of two opposed visions of social reality—the constrained and unconstrained visions. The constrained vision—corresponding to Scruton’s ‘we’ as opposed to the individual ‘I’ of the auteur architect like Le Corbusier or the collective ‘I’ of the despot like Hitler or Stalin--seeks to improve life and ameliorate problems within the recognized constraints imposed by our history and human nature. It sees social decision-making in terms of trade-offs rather than solutions, of the will of millions of individuals relating to each other through neighborhoods, social networks, and markets rather than the will of experts and elites, as accountable directly and indirectly to those masses (hence, socially as well as politically democratic). Like English and American common law, it values the precedents and traditions built up through negotiations and compromises over generations and holds kings and presidents as accountable under the law.

The constrained vision of Sowell or the ‘we’ of Scruton is not cynical or comprehensively pessimistic. But it is modest in its assumptions about the capacity of human beings to remake their society from scratch according to their own desires. Or as Dr. Johnson expressed it,

How small, of all that human hearts endure,
That part which laws or kings can cause or cure.

That planners and utopians have dictatorial, even totalitarian tendencies is not news, though it is too little appreciated. Marx saw in his Third Thesis on Feuerbach how such thinking divides society into two parts, one of which is superior to society. Marx’s own vision, as Sowell argues, is constrained and anti-utopian with respect to the past, but unconstrained and utopian with respect to the future.

The force of Scruton’s argument lies in the detail and concreteness with which he specifies these dangers in every aspect of life, not only in totalitarian regimes like Nazi Germany or the Soviet Union, but also as destructive forces in the democratic West. He points to the violence and destructiveness of French and Russian revolutionaries, to how the first act of such revolutionary elites is to destroy all the institutions of the old society and especially the rule of law that might hold them accountable.

But he also describes in chilling and vivid detail the bizarre grip of the EU bureaucracy on the once democratic and sovereign nations within its orbit. He shows how hundreds of thousands of regulations are issued at an accelerating rate by an unaccountable bureaucracy that cannot be held accountable and whose many mistakes cannot be rectified through democratic processes. Once adopted, those measures cannot be repealed by the nations involved. By the founding treaties of the EU, measures that centralize control in the EU cannot be reversed without constitutional change or leaving the Union altogether. When the Irish electorate rejected the Lisbon Treaty, the bureaucracy merely requested that citizens should vote again. Scruton shows how brutally the bureaucrats sweep away the customs and traditions of centuries, in the process destroying, for example, family farming and the countryside of Romania. He describes how a European directive requiring the presence of a qualified veterinarian at every abattoir led to the closing of most local abattoirs in England, requiring that cattle be taken much greater distances to be slaughtered, so that when disease did break out it spread across the country instead of being localized.

Scruton is particularly scathing in his account of modern architecture, with its scorn both for history and tradition and for the wishes of the people who were to live in and around its brutal structures. Le Corbusier, a key modern architect whose megalomaniac plans are still studied reverently in architecture schools, comes in for particular scorn. The man planned to destroy all of Paris north of the Seine and to rebuild it according to his own design for how people in the twentieth century ought to live. One of his projects, partly carried out, involved the destruction of Algiers. Scruton writes:

The modernist pioneers were social and political activists, who wished to squeeze the disorderly human activity that constitutes a city into a utopian straitjacket…. Le Corbusier’s project to demolish all of Paris north of the Seine and replace it with high-rise towers of glass was supposed to be an emancipation , a liberation from the old constraints of urban living. Those dirty, promiscuous streets and alleyways were to be replaced with grass and trees—open spaces where the new human type, released from the hygienic glass bottle where he was stored by night, could walk in sunshine and be alone with himself. The fact is, however, that Le Corbusier never asked himself whether people wanted to live in this utopia, nor did he care what method was used to transport them there. History (as understood by the modernist project) required them to be there, and that was that (p.143).

Scruton, we can see from this example, has a way with words. His prose is incisive, sharp, and eloquent. He describes how in modernist architecture, “The accumulated wisdom of builders and planners down the centuries was set aside purely on the strength of a ‘best case’ vision of the new ways of living” (p.143). He illustrates his point with an extraordinary account of how such thinking has devastated the cities of the Middle East. Indeed, he argues that if we are seeking an explanation of Islamism, this architectural and planning modernism is the place to start.

Another twist to Scruton’s anti-utopian argument is that the self-image of the progressive elite as more advanced than the masses whose lives they want to manage, is itself illusory. An important aspect of the book is the effort to explain the resistance of these fallacies to reason or evidence. They are, he argues, residues of an earlier stage of human development, one that still holds value in emergencies, but is destructive in times or conditions of peace. There is an implied analogy here to the fight-flight response—once essential for daily survival, but now often dysfunctional as a pattern of intensified arousal in conditions that do not require it.

Scruton appeals, in contrast to the thought-experiments of Rawls or Locke on which social contract theory is built, to the nature of tribes or hunter-gatherer bands as they actually existed. This was the long prehistory before conditions existed for the emergence of societies of unrelated strangers who found ways to live side by side through negotiation and compromise in consensual communities…or cities. In a band of hunters and gatherers that was in constant danger, pursuing and holding on to territory in the face of human and other threats from the outside, survival depends on the collective ‘I’—submission of all to the goals and strategy of a leader. There is no place for worst case scenarios or competing approaches when the band must unite behind its leader or die. The same is true in wartime—which is perhaps why utopias like Edward Bellamy’s Looking Backward or Plato’s Republic are deeply undemocratic and organized top down along more or less military lines.

The importance of writers like Burke and Hayek (and he could have added Marx), in Scruton’s view, lies partly in their understanding of how human settlement, where people produce rather than find their food and other resources, changes everything. “They are pointing,” Scruton says of Burke and the Austrian economists, “to the emergence, in historical societies, of a new kind of rationality—not the ‘I’-rationality of a leader and his plans, but the ‘we’-rationality of a consensual community” (p.210). From this perspective the emergence of a society of institutions and laws--built over generations out of the experience of living together in a society in which individuals and families, as well as corporations, may have goals, but not society as a whole, except in war—is a tremendous but fragile human achievement. Utopians, comprehensive planners and architects, anarchists, terrorists and Islamists all in their different ways threaten that achievement, seeking to undermine or destroy it and to start from scratch.

The Pleistocene mindset of Islamists, like that of the Bolsheviks before them, aims at “entirely destroying the forms of settled government.” Terrorism, thus understood, is a refuge from settlement and a return to the all-commanding ‘I’. It seeks the destruction of settled traditions, laws, practices, and institutions, and in their place the building of a new society designed for (but not by) a new human being. In this sense at least, the Islamists, despite their hatred of modernism, are as modern a phenomenon as Le Corbusier.

The tabula rasa vision of the human being—found in notions of constructing a new “socialist man” or a new human type or, in its weirdest manifestation yet, in a transhuman type that is seen as replacing humans with cyborgs—casts aside those compromises and constraints that previously shaped us. Such indeed was the spirit of the Sixties, with concepts of freedom that wrecked—at least for the poor—the institutions of marriage and fatherhood, social patterns of sexual restraint and responsibility, and many other institutions and traditions that reflected the collective wisdom of generations.

In Scruton’s view, then, the fallacies he describes are rooted in the material needs of hunter-gatherer bands, where everything depends on the will and decisiveness of the chieftain—the leader’s collective ‘I’ is at the same time the ‘we’ of the community. One reason that the fallacies are so impervious to refutation is that they are “not new additions to the repertoire of human madness but the residues of our forefathers’ honest attempts to get things right…thought processes that were selected in the life and death struggles from which settled societies eventually emerged” (p.203). The liberal, optimistic, progressive thinking is not, from this perspective, an advance on the ways and customs of the unenlightened masses, but a regression to more primitive ways of thinking. Scruton’s purpose is to defend the world of compromise and half measures, love, friendship, irony, and forgiveness from the Pleistocene mindset of the enlightened that would sweep them all away.

Empowerment in the Bureaucratic-Professional State
Some of Scruton’s most effective rhetorical shafts are aimed at experts and professionals who, basing themselves on a stock of knowledge and expertise that is largely bogus, usurp the role of families and communities and undermine their capacity to resolve their own problems. In this respect his critique is congruent with that of other critics of the bureaucratic and professionalized social services. For example, in The Careless Society: Community and Its Counterfeits, John McKnight (1996) shows how competent communities have been invaded and colonized by professionalized services—often with devastating results. (See McKnight’s classic essay, “John Deere and the Bereavement Counselor.”)

The patch approach to neighborhood-based social services and the restorative justice approach to family group decision making were attempts to roll back professional domination of poor families (Adams & Nelson, 1997; Burford & Hudson, 2000). They were efforts, in policy and practice, to work in partnership with those directly involved in the problematic situation. They sought to empower the people closest to the situation to draw on their own cultural and individual wisdom and resources to find ways to keep children safe. The aim of these efforts is to strengthen families and communities, through sharing but not abdicating the state’s obligation to protect children from maltreatment.

In this area, Scruton has a brief and provocative, though not nuanced discussion of a typical child protection scandal in the U.K. known as the Baby P case, where a child died who was already known to the authorities and their professionals. The inquiry that followed called for retraining social workers, more expertise, and more funding of services.

For Scruton the area of child welfare is one where the claimed expertise of the professionals is phony. Citing Baskerville’s (2007) critique, Taken Into Custody, Taken into Custody: The War Against Fatherhood, Marriage, and the Family he says: “Examine their expertise, however, and whence it derives, and you will discover a mish-mash of amateur sociology, left-wing dogma and routinized anti-family rhetoric” (p.174). The inquiry’s recommendations reflect the understandable but diversionary tactic of shifting the blame to whatever can be readily blamed, to whatever responds to blame. (He explains much anti-Americanism, within and outside the United States, on this convenient displacement strategy of transferred blame.)

His argument is that this kind of inquiry and recommendation ignores the real forces that created the modern problem of child abuse. It is much easier to retrain social workers or change their practices than to restore the institution of the family. So what is needed, the experts averred, was “more of us, more planning, more supervision, more ways of preventing this society-wide disorder through the intervention of a benevolent state” (p.173).

Citing figures from research in the UK to the effect that children are vastly more likely to be abused in the homes of mothers with a live-in boyfriend or stepfather than in an intact family, Scruton says, “Actually what Baby P needed was a father, and the smallest dose of pessimism would have pointed this out” (p.173). To think in this way, however, is to run up against “one of the fundamental prejudices of the time: the prejudice that the new forms of domestic life brought about by easy divorce and the sexual revolution are unalterable and unquestionable. Child abuse is not a universal social disorder, for which the state bureaucracy and its experts are the cure. It is the direct result of the delegitimization of the family, often carried out by those very experts. Meanwhile, the state has connived in the dissolution of the marriage tie, and has routinely subsidized, through the welfare system, the arrangements (including live-in boyfriends) that expose children to danger” (pp.173-174).

Problems of Curmudgeonly Writing
Scruton’s prose is witty, clear, and eloquent, always a pleasure to read even when one disagrees with him. His curmudgeonly tone comes from the bitter experience of a brilliant scholar whose academic career in England was blighted for most of its span because his colleagues found his views—those of a Burkean conservative—unacceptable and too far beyond the liberal-radical consensus of the academy (outside the sciences, anyway). The fury with which progressive thinkers respond when the fallacies in their thinking are pointed out has been visited on Scruton’s head in print and in the harshest tones.

It is natural in these circumstances that he would conclude that “the argument of this book is entirely futile. You may enjoy it and agree with it, but it will have no influence whatsoever on those whom it calls to account” (p.3). How could he conclude otherwise after a lifetime of collegial abuse? (This is not to deny the compelling case Scruton makes that the fallacies he examines are indeed resistant to correction, without regard to the author’s personal experience.)

That perception of futility, however, as well as the large scope of the argument compared with the modest size of the book, creates its own limitations. Scholarly rigor, careful documentation of the examples and fair consideration of objections and alternative arguments must seem hardly worth the trouble since, in any case, those who comprehensively disagree with the author will not themselves be open to argument.

So Scruton’s dismissal of multiculturalism, progressive education, postmodern gobbledygook, and the like are witty and a delight to read, but do not seriously engage the advocates of those follies. His account of how utopian notions of “education for equality” in the UK succeeded only in destroying opportunity for gifted working-class children and ensuring as nearly as possible that students did not learn anything, is fun to read. His view of education experts with their “agenda that was uniformly egalitarian, child-centered and knowledge-averse,” (p.172) and their disastrous effects on education is scathing and witty, but probably not compelling to an educationist.

Some of his examples are also tendentious and do not consider alternative explanations—for example, he sees the housing crisis as arising from government interference in the market with the aim of encouraging people who could not afford them to buy homes and banks to lend them the money to do so. Scruton does not consider the alternative (or is it complementary?) explanation that the problem was too little government regulation rather than too much.

He makes complex economic arguments but dismisses economics as a phony discipline. His argument is at some level theological, but he dismisses theology as another area of phony expertise—because we cannot know anything about God. He does not draw the obvious conclusion that theistic religions are ipso facto bogus, or that the theological conclusions about God in the Nicene and Apostles’ Creeds that resulted from centuries of theological study and debate are worthless. Nor is it clear what he means by using the word God as if God existed but we could not know anything about the subject of that existence—except, presumably that God exists. His position seems not to derive from the tradition of negative theology that addresses only what we do not know about God or that of apophatic mysticism that emphasizes the unknowability of God. Rather, it seems to reflect a characteristically Anglican agnosticism, of one who enjoys the traditions and rituals of the Church of England but who is not sure he believes a word of it. (Indeed, Scruton’s curious theology invites a whole separate discussion.)

Most seriously, Scruton pays little or no attention to the most obvious questions his critique raises. Are tradition and custom so benign? What about slavery or female genital mutilation or suttee? These are the standard questions raised about multiculturalism and a cultural/moral relativism that regards all cultures as equal (or equally deserving of respect). Since Scruton has no time for such postmodern or politically correct tendencies, it is surprising that he does not take greater care to explain how his valuing of tradition addresses such questions. It is not that they cannot be addressed. English conservatives like Burke or Samuel Johnson supported the American Revolution and/or opposed slavery without difficulty or inconsistency. But Scruton does not take the trouble to anticipate such objections or explain his position to skeptical readers.

At times, Scruton’s manner is reminiscent of a father who provokes his liberal and idealistic children by making provocative remarks he knows the young people will find outrageous. He knows there is nothing he can say that will persuade the younger persons to re-examine their views or look at them with a measure of irony. The elder will not be intimidated or silenced by the usual conversation-stopping insults (right wing, racist, sexist, bourgeois, etc.) but thinks it pointless to defend himself against them. However seriously misguided he thinks the young are, and however disappointed in their failure to take seriously the fruits of his knowledge, experience, and wisdom, he consoles himself by getting a rise out of them and a chuckle from the other grown-ups.

But the curmudgeon stance makes it too easy for critics like George Scialabba (at http://www.georgescialabba.net/mtgs/archive-by-date.html) to dismiss the book as a partisan rant. That is a shame. Scruton is a brilliant author—philosopher of ethics and aesthetics, critic of music, art, and architecture, commentator and polemicist—of extraordinary depth and range. His work challenges received wisdom in the social sciences and humanities. His critiques, even when lacking the full apparatus of German scholarship, are serious attempts to offer a coherent and comprehensive alternative to the dominant thinking in the academy, arts, and media.