Reasons not to be cheerful: one, two, three.

In my forthcoming annual lecture I offer an explanation for our apparent inability to make progress on challenges most people would like to see addressed; like providing dignity and decent care for all old people, increasing social mobility or achieving a more sustainable model of economic growth. I argue that there are three basic sources of social power, hierarchical authority most often associated with the state, solidarity most often associated with community, and individualism most often associated with the market.

The problem is that in different ways, and together as a whole, these sources of power have dried up. I explore the problems facing hierarchy (incompetence and low public trust for example) and solidarity (as a consequence of diversity and social polarisation), and argue that while individualism is powerful, it is also narrow and in many ways self defeating.

However, the problem with individualism is not so much that it has, under its own volition, mutated into something malign. Like the other social forces, the dangers of individualism are the flip side of its strengths, the latter including creativity, ambition and drive. However – as we saw in the financial sector - individualism unchecked by wise and trusted authority or the binds of solidarity and social responsibility tends towards selfishness and irresponsibility.

The characteristics of a well functioning society, organisation or strategy is the balance of top down (doing what you’re told), lateral (doing what others do), and bottom up (doing what you choose) ways of seeing and pursuing change. When one dimension fails not only is its power for constructive change lost, but it leads to the other dimensions being overloaded.

I came across an interesting case study of this phenomenon reading reviews of a new book causing much debate in America: The Twilight of the Elites by Chris Hayes. The book is an attempt to explain the multiple failings of authority in the US in the first decade of the 21^st century (ranging from Enron and Worldcom to Katrina and Iraq, to Catholic priests and sports administrators).

I have read only the reviews (we will try to lure Mr Hayes over to the RSA some time), but I find it easy to believe the book’s account of how meritocracy has failed, placing particular emphasis on how elites close ranks and self perpetuate. Hayes’ solution is more controversial, although he is far from alone in advocating it. In essence he argues that meritocracy can only work within limits to overall levels of inequality. When society is highly unequal the rewards for staying at the top and the fear of dropping down make it impossible for all but the most socially responsible elite member not to use any means possible (including exploitation and gaming) to try to stay there and get their kids to join them.

The point is this: the individualistic ideal of meritocracy (remembering that social democrat Michael Young used the word disparagingly when he first defined it) only works alongside the solidaristic principle of social justice and, furthermore, that when solidarity fails to set limits to individualism the consequence is an incompetent and self-serving hierarchy.

Many American commentators have wondered aloud whether it would ever be possible to forge an alliance between the Tea Party and the Occupy movement, given that both share a profound hostility to the elite they believe are misruling their country. Given the degree to which both movements are driven by anger, the attribution of blame and a tendency to see conspiracies, I’m not sure this is either likely or to be desired. It is the system that is failing, something reflected in rising social pessimism, but the partisan nature of political campaigns means they only focus on one aspect of that failure. This is why when we listen to their message we often have the odd feeling of simultaneously agreeing with the critique while rejecting the overall analysis.

The problem is not a lack of leadership, nor a lack of social justice and responsibility, nor an inadequacy of personal responsibility and enterprise, but all these together acting on each other to create a deteriorating stasis. The answer is a counterintuitive combination of big thinking (whole system reform with implications for all of us) and moderation. The problem is functional (power to) rather than oppressive (power over).

Perhaps the system will correct itself (history provides plenty of grounds for both optimism and pessimism), but if we are waiting for a movement or a leader capable of articulating, let alone winning, an argument as profound but nuanced then looking at politics here, in Europe or – most depressingly – America, there are few reasons to be hopeful.