The Wisdom of Crowds

In this fascinating book, New Yorker business columnist James Surowiecki explores a deceptively simple idea: Large groups of people are smarter than an elite few, no matter how brilliant—better at solving problems, fostering innovation, coming to wise decisions, even predicting the future.

Who wants to be average? To be average is to be commonplace and unexceptional. It conjures up the mediocre or banal – the consensus, the mass, the run of the mill. But here’s the rub: in the right circumstances, the average is the best place to be. Take a competition based on guessing something, such as the number of jellybeans in a barrel or the weight of a cow. In most cases, the average of a crowd’s guesses is more likely to be accurate than those of its individual members. Although it may be counter to our intuitions, the masses can be smarter than the solitary expert. So much for mediocrity.

That insight is at the heart of James Surowiecki’s intelligent book. What he lays out is a valuable counter-argument to the contempt for the crowd that dates back to the Victorian era. Considering the widening of the franchise and the rapid advancement of the working classes, writers and philosophers on both sides of the Atlantic worried about the effect of unleashing the masses upon the ballot box. Writers such as Thomas Carlyle expected a repeat of the horrors that followed the French revolution in 1789 or the barricades of 1848: mass unrest and disorder, with the newly enfranchised crowd seizing wealth and power. Luckily, Carlyle was wrong, as he was on so many other things. Instead, the interesting question is why the era of mass enfranchisement has seen the less well-off members of society so frequently vote against their own economic interests. Given the breathtaking inequality that exists in many democracies, how is it that the average doesn’t choose parties backing economic redistribution?

The answer to that question reveals the limitations on the wisdom of crowds. The average of a sufficient number of guesses will get the number of jellybeans right. But supplying distracting or misleading information can easily skew the individual’s responses. Collect enough people on a street corner staring at the sky, and everyone who walks past will look up. Show a voter enough political broadcasts or commentaries and she may be persuaded to vote against her interests.

Given enough encouragement, a large group may even be convinced that black is indeed white. In 1841 the Scottish writer Charles Mackay published Extraordinary Popular Delusions and the Madness of Crowds, a compendium of the cacoethes to which societies succumb, including the Dutch tulip mania of the 1630s, the South Sea bubble, the crusades and long hair. To Mackay’s list, Surowiecki adds the dotcom boom of the late 1990s, and the less well-known but more amusing bowling bubble of the 1950s, caused by a fanciful calculation that every man, woman and child in the US would bowl for two hours a week, 52 weeks a year. A financial market is a crowd like any other but, to paraphrase JM Keynes, the market can be wrong for longer than you can afford to be right.

Even so, Surowiecki’s book is an antidote to the notion that a camel is a horse designed by a committee. The example of the spread of Linux open-source software shows the intelligent crowd in action, able to produce reliable and cheap computer software compared with the alternatives from companies such as Microsoft.

Surowiecki recognizes the weaknesses of groups in situations where they would be expected to show their mettle. The Nasa mission team that discussed the risks facing the space shuttle Columbia in 2003 failed to see the damage to the shuttle’s wing that caused it to be destroyed on re-entry. Similarly, the intelligence services of the US and Britain failed to consider the possibility that Iraq harbored no weapons of mass destruction. In both cases, the result of “group think” was no better than that of an individual. Even groups of experts work properly only when they allow space for dissenting voices to be considered.

The Catholic church was wise to adopt the office of “Advocatus Diaboli” when considering its candidates for sainthood. A group that merely confirms already held opinions serves no useful purpose. A committee that thinks of nothing but camels will surely design one; a committee that frets about weapons of mass destruction everywhere will imagine them anywhere.


Previous Story

Google fires Margaret Mitchell, founder and co-head of the AI ethics team

Next Story

Sandile Cele discovers an innovative way of growing the new Covid-19 variant