Societal Security

Humans have a natural propensity to trust non-kin, even strangers. We do it so often, so naturally, that we don’t even realize how remarkable it is. But except for a few simplistic counterexamples, it’s unique among life on this planet. Because we are intelligently calculating and value reciprocity (that is, fairness), we know that humans will be honest and nice: not for any immediate personal gain, but because that’s how they are. We also know that doesn’t work perfectly; most people will be dishonest some of the time, and some people will be dishonest most of the time. How does society — the honest majority — prevent the dishonest minority from taking over, or ruining society for everyone? How is the dishonest minority kept in check? The answer is security — in particular, something I’m calling societal security.

I want to divide security into two types. The first is individual security. It’s basic. It’s direct. It’s what normally comes to mind when we think of security. It’s cops vs. robbers, terrorists vs. the TSA, Internet worms vs. firewalls. And this sort of security is as old as life itself or — more precisely — as old as predation. And humans have brought an incredible level of sophistication to individual security.

Societal security is different. At the tactical level, it also involves attacks, countermeasures, and entire security systems. But instead of A vs. B, or even Group A vs. Group B, it’s Group A vs. members of Group A. It’s security for individuals within a group from members of that group. It’s how Group A protects itself from the dishonest minority within Group A. And it’s where security really gets interesting.

There are many types — I might try to estimate the number someday — of societal security systems that enforce our trust of non-kin. They’re things like laws prohibiting murder, taxes, traffic laws, pollution control laws, religious intolerance, Mafia codes of silence, and moral codes. They enable us to build a society that the dishonest minority can’t exploit and destroy. Originally, these security systems were informal. But as society got more complex, the systems became more formalized, and eventually were embedded into technologies.

James Madison famously wrote: “If men were angels, no government would be necessary.” Government is just the beginning of what wouldn’t be necessary. Currency, that paper stuff that’s deliberately made hard to counterfeit, wouldn’t be necessary, as people could just keep track of how much money they had. Angels never cheat, so nothing more would be required. Door locks, and any barrier that isn’t designed to protect against accidents, wouldn’t be necessary, since angels never go where they’re not supposed to go. Police forces wouldn’t be necessary. Armies: I suppose that’s debatable. Would angels — not the fallen ones — ever go to war against one another? I’d like to think they would be able to resolve their differences peacefully. If people were angels, every security measure that isn’t designed to be effective against accident, animals, forgetfulness, or legitimate differences between scrupulously honest angels could be dispensed with.

Security isn’t just a tax on the honest; it’s a very expensive tax on the honest. It’s the most expensive tax we pay, regardless of the country we live in. If people were angels, just think of the savings!

It wasn’t always like this. Security — especially societal security — used to be cheap. It used to be an incidental cost of society.

In a primitive society, informal systems are generally good enough. When you’re living in a small community, and objects are both scarce and hard to make, it’s pretty easy to deal with the problem of theft. If Alice loses a bowl, and at the same time, Bob shows up with an identical bowl, everyone knows Bob stole it from Alice, and the community can then punish Bob as it sees fit. But as communities get larger, as social ties weaken and anonymity increases, this informal system of theft prevention — detection and punishment leading to deterrence — fails. As communities get more technological and as the things people might want to steal get more interchangeable and harder to identify, it also fails. In short, as our ancestors made the move from small family groups to larger groups of unrelated families, and then to a modern form of society, the informal societal security systems started failing and more formal systems had to be invented to take their place. We needed to put license plates on cars and audit people’s tax returns.

We had no choice. Anything larger than a very primitive society couldn’t exist without societal security.

I’m writing a book about societal security. I will discuss human psychology: how we make security trade-offs, why we routinely trust non-kin (an evolutionary puzzle, to be sure), how the majority of us are honest, and that a minority of us are dishonest. That dishonest minority are the free riders of societal systems, and security is how we protect society from them. I will model the fundamental trade-off of societal security — individual self-interest vs. societal group interest — as a group prisoner’s dilemma problem, and use that metaphor to examine the basic mechanics of societal security. A lot falls out of this: free riders, the Tragedy of the Commons, the subjectivity of both morals and risk trade-offs.

Using this model, I will explore the security systems that protect — and fail to protect — market economics, corporations and other organizations, and a variety of national systems. I think there’s a lot we can learn about security by applying the prisoner’s dilemma model, and I’ve only recently started. Finally, I want to discuss modern changes to our millennia-old systems of societal security. The Information Age has changed a number of paradigms, and it’s not clear that our old security systems are working properly now or will work in the future. I’ve got a lot of work to do yet, and the final book might look nothing like this short outline. That sort of thing happens.

Tentative title: The Dishonest Minority: Security and its Role in Modern Society. I’ve written several books on the how of security. This book is about the why of security.

I expect to finish my first draft before Summer. Throughout 2011, expect to see bits from the book here. They might not make sense as a coherent whole at first — especially because I don’t write books in strict order — but by the time the book is published, it’ll all be part of a coherent and (hopefully) compelling narrative.

And if I write fewer extended blog posts and essays in the coming year, you’ll know why.

Posted on February 15, 2011 at 5:43 AM137 Comments


Lasse February 15, 2011 6:16 AM

To further substantiate the introduction to this blog post, you could look into the works of Danish philosopher and theologian Knud Ejler Løgstrup – especially “The Ethical Demand”.

His ontological ethics is a third alternative to the utilitarian and deontological approaches claiming that we are essentially born good and that we are always interdependent on each other – we hold the fortunes of others in our hands everytime we interact.

Could provide a background for arguing what to do with the exceptions from the rule – those who abuse others’ trust…

John F. Fay February 15, 2011 6:18 AM

Definitely sounds like it will be a very good book. I was going to recommend Robert Axelrod’s “The Evolution of Cooperation” as a source but with your reference to the Prisoner’s Dilemma it sounds like you’re already onto it.

  • John F. Fay

Kerrek SB February 15, 2011 6:25 AM

Great, best of luck with the book!

One thing about money and taxes: I think that first and foremost those two are means of management rather than security. People simply couldn’t remember how much money they have and trade efficiently, or contribute effectively to public spending. Imange, each time you build a bridge you have to ask everyone to give the builders a part of their virtual money. Combinatorial explosion would make this infeasible. Even angels need money (unless they’re the ethereal kind who don’t need bread or bridges), especially if there’s a whole city of them.

Hugo February 15, 2011 6:26 AM

Nice!! I see a Christmas present coming.

Do you have an Amazon item id for us so we can pre-order? 🙂

Grumpy February 15, 2011 6:34 AM

That is probably the most important book I’ve heard about in a long, long time. Interesting too, but important in the sense that if we have no idea why we’re doing something, it’s hard to know how much and how long we have to do it. Get it ready and I’ll buy quite a few copies. This idea needs to get around. 🙂

Ben Fowler February 15, 2011 6:56 AM

There’s no shortage of interesting ideas for this problem. You definitely have a good start, and I, too, was immediately reminded of Axelrod’s “The Evolution of Cooperation”. It’s easy to take a few thoughts and integrate them into that work, to suddenly find you’ve explained a large part of individual human and societal behavior in ways no one else has quite considered.

Hans E Peterson February 15, 2011 6:56 AM

A little input: In game theory, applied on human behavior, it seems that the most successful strategy ever is “hit once when hurt/betrayed, then trust again”. All other strategies seem to give worse results when applied to both theoretical and real environments. It may explain what you find strange and unique for humans on this planet, and it may be one imortant reason for our relative success among other species.

asdf February 15, 2011 6:57 AM

@Kerrek SB: Angels don’t need money, they understand that in order to make use of the bridge, they need to cooperate to build it 🙂

Harry February 15, 2011 7:00 AM

If humans were angels, communism would work.

Sidenote: Garrett Hardin, who created the phrase “the tragedy of the commons” later wished he’d called it “the tragedy of the unmanaged commons.” (“Commons Sense:
Why it still pays to study medieval English landholding and Sahelian nomadism,” The Economist. 31 July 2008,, accessed 15 February 2011.)

If you haven’t found it yet, I strongly recommend the article and the comments to you. In addition to being well written it includes counter-arguments (ie, that a common need not be tragic) and references for more reading on both sides.

wkwillis February 15, 2011 7:07 AM

I was taught in security guard training that 30% would steal if they wouldn’t be caught, 30% would steal if they wouldn’t be punished, and 40% would not steal, period.

Trichinosis USA February 15, 2011 7:08 AM

This is fascinating, but I think to ensure you cover all possible bases, you need to study how this problem is handled worldwide. This is as much about anthropology as anything else, and not all societies handle the issue of security the way the US does – especially post-9/11. We are currently regressing to an almost feudal system of security, and that’s not a step forward at all.

Jay from BKK February 15, 2011 7:12 AM

@Rob: “why we routinely trust non-kin (an evolutionary puzzle, to be sure)”

Not much of a puzzle. One part herd instinct (I don’t have to outrun the bear, I only have to outrun you…), and one part hard-wired understanding that others may have something we lack even if we don’t yet realize that we lack it.

As a human you can get something of value (tangible or otherwise) from almost any other human, even if only by negative example. Thorg may not give you his slingshot nor even drive off a predator to your benefit or share his meat, but you may pick up a few pointers, or just get a life-extending belly laugh from watching him use it backwards.

Whether you get that value by methods society artificially judges as good or ungood is a matter solely of context and in a thousand years will be meaningless as the context will be vastly different.

Adrian Scott, Ph.D. February 15, 2011 7:50 AM

If I read what you’re writing correctly, it sounds like your model may have a major flaw, but hopefully I’m wrong.

It sounds like it is assuming a constant composition of the society membership, and is focused on a single-society/single-country worldview. There’s a lot of macroeconomics that makes bad assumptions about mobility between countries/societies which lead to heavily flawed models (particularly when addressing certain problems).

[ On the dishonest minority side of things, I think there’s extensive research about various factors that can influence an individual’s honesty; i hope that works in the model. ]

I’m a huge fan of your work and am looking forward to the book. I hope these comments are helpful in stimulating thought.

Joe February 15, 2011 7:54 AM

“why we routinely trust non-kin (an evolutionary puzzle, to be sure)”

I think it is less of an evolutionary puzzle if we think of the need for genetic diversity.

Suzanne February 15, 2011 8:10 AM

Exactly, Joe. You can be social and rely on trust and altruism to keep as many individuals alive as possible in order to increase the gene pool, or you can be an animal that lives more individually and rely on an enormous birth rate.

Phillip February 15, 2011 8:16 AM

Another form of society security a company employs is forcing employees to take their vacation. In fact, all such Internal Audits are a form of societal security, no?

noble_serf February 15, 2011 8:36 AM

Just offering encouragement.

I saw you on a Smithsonian Channel show last night (about internet security). I was trying to explain to my better half that I read your blog often and you’re a smart guy. It was funny, she said, “What? A guy with a pony tail?”

Just a cultural example of how we judge others (she’s not from western culture, so an older smart white guy with a pony tail is a bit odd to her)

Jim A February 15, 2011 8:40 AM

Well there is some game theory thinking that if you’re going to have repeated dealings with somebody, dealing with them honestly and forthrightly (or at least not get caught cheating them) is a long-term winning strategy. Conversely, if you’re never going to see them again,* you might as well ruthlessly take advantage of them. So to some extant, we are evolutionarily hard wired to regard and treat those inside our “society” very differently from those outside of it. Outside of combat, this in/out decision is rarely so black and white but rather shades of grey. We will do the most that we can for our immediate family, less for those with whom we share other bonds, and least of all for foreigners of different religions and social classes. Of course different people asses those ties of kinship differently, and figuring out how somebody ELSE regards your level of kinship is a critical survival skill.

*Reputations mean that the question isn’t really whether you’ll deal with them, rather whether you’ll deal have to deal with somebody who will listen to their opinion of you.

MarianC February 15, 2011 8:46 AM

Society didn’t go from primitive hunter-gatherers to modern civilization overnight. The evolution of social behavior has taken place over thousands of years of development. For an excellent source of insight, I recommend “Why The West Rules—For Now”, by Ian Morris.

Clive Robinson February 15, 2011 9:00 AM


You first have to explain that there can be no good without evil and vis versa that is evil is the measure of good and good the measure of evil, neither can exist without each other.

Therefore no Angels without fallen Angels.

Part of mankind is that it measures it’s self and strives to be better. Thus the concept of what is good bad or acceptable changes with time and thus measures society.

It appears also that mankind cannot look up to it’s self so we invent Gods by which we measure ourselves.

However we have the need of secrecy, it is part of competative or mating advantage. That is if we have a source of plant food then we tend to hide it from others in our group so that we can exploit it for our own benifit as it ripens. However animal food or flesh rots quickly and there is little competative advantage in keeping it from others of our group. In fact there are competative advantages in sharing it amongst all as it raises an individuals status within the group.

Desmond Morris hypothesized that “meat eating is social” and “Vegatable eating is anti social” even to day. That is we see many people eating meat together (ie go for a burger, or invite to a barbecue or meal with a meat center piece) but we seldom see people just eating fruit or vegtables together.

BobW February 15, 2011 9:16 AM

Be very careful that you to not make the mistake of assuming the entire world is Northern European societies. We have high trust societies. Not everyone does.

jay February 15, 2011 9:58 AM

Your assertion that Government wouldn’t be necessary if people were perfectly honest (“angels”) has a flaw. Government (or some equivalent) would still be necessary because even given perfect people they still have limited knowledge. Government would be, and is, used to manage resources and provide services more efficiently than a single individual can.

Dr. I. Needtob Athe February 15, 2011 10:19 AM

Atheists are often attacked with the dubious assertion that “you can’t be good without God”; that without religion there is no source of moral values. It’s argued by religious people that survival of the fittest and natural selection can’t explain morality, therefore it must come from God.

It sounds like this book might have potential to shed some light on that controversy.

AlanS February 15, 2011 10:48 AM

A lot of what you outline here sounds like basic anthropology where reciprocity, accountability, the social uses of risk, etc. are bread and butter topics. Sociologists also have much to say on these topics.

When things are on a smaller scale, institutions are simpler, and relationships are more multiplex, it’s easier to hold people accountable for their actions. And you don’t have to go back in time to a “primitive society”–a term I’d skip–to find these conditions. In a big city where there’s more potential for anonymity or there are lots of short-term, single-stranded relationships it is often not easy to hold people accountable.

Anyway there are so many texts on these topics in the social sciences it’s hard to know where to start. For a nice text on the anthropology of strangers and urban risk Sally Merry’s “Urban Danger: Life in a Neighborhood of Strangers” is insightful.

Interesting that you quote Madison on the topic. Madison was educated at the College of New Jersey, now known as Princeton, a Presbyterianism institution. As such he was stepped in the writings of the Scottish Enlightenment: Hume, Smith, Ferguson, etc., all of whom thought and wrote deeply about the political and social practicalities of modern civil society.

Smith has lots of interesting stuff to say about risks, reciprocity, etc. However, if you take a look at Wealth of Nations rid your mind of all the nonsense written by economists, pols, journalists and others who cite the book on an almost daily basis because most of them grossly misrepresent it’s contents–most economics Nobelists no exception.

LandruBek February 15, 2011 10:58 AM

Please don’t call it “The Dishonest Minority” — it sounds racist.

Also I want to underline what BobW said above: there is more than one equilibrium point for trust. Western society has found a workable point where we mostly trust others but pay the price of having to behave trustworthily. In other societies (I’m thinking of Russia), we mostly distrust others and are largely free of the burden of having to behave trustworthily.

NobodySpecial February 15, 2011 11:12 AM

@wkwillis – it’s more complex than that.
100% would cheat on their taxes – even churches do that.

99.9% would cheat on parking meters, transit fares and other ‘government’ fees

A majority would steal from a casino, bank or large corporation if they could get away with it.

A small percentage would steal from a little old lady.

Petréa Mitchell February 15, 2011 11:32 AM

Armies? Easy: lots of wars have been started for honest reasons. And no amount of talking is going to resolve a situation where this is the last water hole left in a drought, and it’ll only sustain 50 people, and your tribe plus my tribe equals 75 people.

Petréa Mitchell February 15, 2011 11:45 AM

Hans E Peterson:

The one problem with that strategy is that it can wind up in a retaliatory loop when confronted with a similar one.

The same game theory competition that named that strategy “best” found an even better one some years ago: to stop the cycle of retaliation, forgive occasionally. Last I heard, turning the other cheek 30% of the time was found to be optimum.

mcb February 15, 2011 11:50 AM

Excellent topic. Evolutionary selection for empathy, taboo, reciprocity, cooperation, and altruism – aka ethics and morality, at the neurobiological level is fascinating stuff. It’s all the more amazing because we seem to be getting better at it. I’ll be interested in how you deal with counter examples of small group ethnocentrism, xenophobia, in group/out group differentiation, homicide rates among hunter-gather societies, etc.

Pinker touched on it in his TED talk: On the Myth of Violence

An important slide referred to but not seen in his video is drawn from the work of Manuel Eisner

An old favorite of mine Of Arms and Men by Robert L. O’Connell addresses the issue of inter- and intra-specific conflict in paradigm-shifting detail.

I found this just now

Please keep us posted.

Petréa Mitchell February 15, 2011 11:56 AM


Natural selection recognizes no need for genetic diversity with a species; indeed, it often works against it.

Once upon a time, a mutation developed among pre-humans that conferred enhanced social skills, and those abilities led to enhanced survival and breeding, which spread the genes more. That’s all there ever is to evolution: a mutation appears, it turns out to be useful, it allows those with it to outbreed those without. No “puzzle” about it.

DMW February 15, 2011 12:11 PM

Before re-inventing Social Contract theory, please make sure to read Locke, Rousseau, and the other Social Contract philosophers. Turns out people have thought about this before.

fraac February 15, 2011 12:13 PM

I think you risk rationalising a purely natural phenomenon. People trust others higher up their social hierarchy, or anyone who can fake it. End of story.

Petréa Mitchell February 15, 2011 12:14 PM

Clive Robinson:

“Desmond Morris hypothesized that ‘meat eating is social” and ‘Vegatable eating is anti social” even to day. That is we see many people eating meat together (ie go for a burger, or invite to a barbecue or meal with a meat center piece) but we seldom see people just eating fruit or vegtables together.”

Well, he’s wrong. Food-sharing rituals have been around as long as human beings and very few of them absolutely require meat.

You and I see people making meat the centerpiece of their food gatherings because meat is tasty, cheap, and easy to get for the people around us. If meat were impossible to get, people would still eat together.

Bruce Schneier February 15, 2011 12:16 PM

“Even angels need money (unless they’re the ethereal kind who don’t need bread or bridges), especially if there’s a whole city of them.”

Agreed, but they wouldn’t need it to be secure in any way. Angels could use Monopoly money, or just keep a running tally in a public database somewhere.

Bruce Schneier February 15, 2011 12:17 PM

“Nice!! I see a Christmas present coming.”

I’d be surprised if it is published by the end of the year. I suppose it’s possible, but I’d be surprised.

“Do you have an Amazon item id for us so we can pre-order?”

I don’t even have a publisher.

Bruce Schneier February 15, 2011 12:19 PM

“It sounds like it is assuming a constant composition of the society membership, and is focused on a single-society/single-country worldview.”

I don’t think I am making this assumption, but I will keep it in mind.

Bruce Schneier February 15, 2011 12:21 PM

“Be very careful that you to not make the mistake of assuming the entire world is Northern European societies. We have high trust societies. Not everyone does.”

Agreed. There’s a lot written on the development of trust in society.

Bruce Schneier February 15, 2011 12:26 PM

“Please don’t call it ‘The Dishonest Minority’ — it sounds racist.”

I honestly never parsed the phrase in that way before: that “minority” refers to a racial minority and “dishonest” is a pejorative term applying to that racial minority.

I’ll watch for that reaction again, but I’m dubious. Does “Senate Minority Leader” sound racist as well?

Bruce Schneier February 15, 2011 12:28 PM

“Before re-inventing Social Contract theory, please make sure to read Locke, Rousseau, and the other Social Contract philosophers.”

Yes, of course. But I’m less interested in the philosophy behind the arrangements and more interested in actual human behavior within those arrangements.

NoOneSpecial February 15, 2011 12:48 PM

We used to be able to kill off those that threatened the group. Society says that is not acceptable anymore – hence we pay dearly and there is no reason anymore not to be dishonest. The punishment for being dishonest, even blatantly dishonest, is increasingly mild in comparison with the damage is causes to the group.

bob!! February 15, 2011 12:54 PM

@jay – when you said “or some equivalent” you turned your argument into a tautology.

If you call an organization that does things that governments do a ‘government’ (or some equivalent) then anything that performs a function that governments currently do fits your “some equivalent” no matter how it’s organized or by whom. What Madison wrote (and Bruce quoted, rather than asserted) is not that we wouldn’t need the functions that governments currently provide (although we certainly wouldn’t need policing or tax collecting), but that we wouldn’t need to band together to create coercive government power in order to keep people from free-riding off the contributions of others.

There are certainly things that individuals or small groups wouldn’t be able to do efficiently on their own. But if you look at the things that (for example) municipal governments do, in the absence of a municipal government, Angels would organize themselves to get them done, in at least two ways that spring to mind.

The first, for small & easily managed things, an Angel recognizes a recurrent need for trash collection and would start collecting trash. Other Angels, being perfect, would, of course, realize how useful this service is to them, and would compensate the trash collector accordingly.

The second, a group of Angels note that certain things would be more efficient if done in an organized manner (a regular schedule of street maintenance, rather than just fixing potholes one at a time). So they would chip in a bunch of resources (more than their fair share, what with being Angels and realizing that all the other Angels will compensate them accordingly for the fact that they put up more resources than they was their fair share).

Alex February 15, 2011 1:46 PM

I doubt you read any comments, but anyway.

The first sentence should be corrected as “Western people have a natural propensity to trust non-kin, even strangers.”

You will never find this trust in developing countries. Afrikan, Asian, Russian, Persian people never trust strangers. If you smile to a stranger in Moscow, you’ll easily get yourself into trouble. If you ask for help, and you don’t look like a person who needs any help, you won’t get any help from strangers.

Unlike in Western countries, there are no self-checkout kiosks in supermarkets in poor countries. There are no book stands near the book shops on the streets, and no unprotected back doors in the private houses. Etc, etc.

Nevada February 15, 2011 2:00 PM

Alex: “I doubt you read any comments, but anyway.”

You somehow managed to miss the seven direct responses by Bruce to comments in this post just above yours.

Richard Steven Hack February 15, 2011 2:18 PM

This is an economic issue. Economics is based on human behavior (see Ludwig Von Mises and the Austrian Economics School). Human behavior in turn is based on our primate evolutionary history.

There are two ways a social entity can function: trade or coercion. Most people invest energy and resources into producing something to trade in order to achieve a monopoly profit. Most of the time they don’t get it, but they do get some profit. Coercion is an attempt to achieve a monopoly profit without actually producing something to trade.

The problem with coercion is that if it is successful, more and more people will invest in it because that’s what people do: invest in those activities that tend to produce a profit and especially those activities that can produce a monopoly profit.

But since coercion does not produce anything, the result of investment in coercion results in a society in which massive amounts of resources are diverted into defense against coercion. The end result is that even those doing the coercion have to expend most of their profit defending against coercion. This turns a productive society into a zero-sum game.

This is why coercion must be restricted in any society of sentient entities. It’s simply a threat to the advancement of that species. There is no need to go into morals or religion to explain this – it’s simply a fact of human economic behavior.

So the question which must be asked is: how does one restrict investment in coercion? The answer is clearly one and one only: proper training and education of the young as to the inevitable long-term consequences of investing in coercion as a behavior.

But this presumes a rational society – which doesn’t exist today or ever in human history. The problem is that as a result of the primate evolutionary background, humans end up generating a lot of conflict which is mostly unrelated to issues of profit and loss. This leads to coercion being the preferred behavior pattern for large numbers of humans.

However, in ancient times, coercion to acquire resources was also rampant. What happened then is that some of those coercers discovered that robbing caravans was not efficient. Like the bandits in the famous movie “The Magnificent Seven”, they learned that it was better to start what we now call “protection rackets”. It started out with bandits demanding tribute from caravans in order to “protect you from the bandits in the region.” Then it evolved into the bandits taking control of the non-coercive producers in villages and towns under the same principle: “You do everything we say and give us most of what you have and we’ll protect you from the bad people inside and outside our borders. And if there aren’t any bad people, we’ll make some based on religion, ethnicity, skin color, whatever.”

Thus was the state born. The state by definition attempts to achieve a monopoly on the use of coercion over a given population. Unfortunately, the world is large, people are diverse, and no state can achieve that monopoly. But they all keep trying to the degree that is reasonable for the size and composition of their populations.

But the end result of this is a state of world society where coercion is seen as profitable – precisely because the state is profitable. And that encourages investment in all kinds of coercion. It specifically encourages those who engage in coercion to attempt to subvert the state – which is usually not hard because those who join the state do so almost invariably because of a predilection for coercion.

This in turn produces a society in which the intelligence and education of the population have to be kept under control because otherwise the members of the state would be exposed for the coercers they are.

The end result is the “anarchy of states” that the world is today. And the rise of crime and the concomitant lack of security in almost every society in the world, as well as the unending state of permanent war.

I’ve just explained everything worth knowing. Bruce can use it in his book if he likes.

Harry February 15, 2011 2:43 PM

About those Angels and how they’d organize and compensate each other:

James P. Hogan wrote an SF book called “Voyage From Yesteryear” in which a society evolved de novo. Their “currency” was respect for competency. If someone liked painting houses he painted houses (with permission of the occupant); if he were very good at it he’d be “rich.” Jobs that no one wanted to do were done by machines.

There were few free riders in this fictional society. Those who did nothing were low in status, poor, and pitied.

Imperfect Citizen February 15, 2011 3:02 PM

Sounds great! I was reflecting that since I was targeted I’ve trusted non-kin and new people less. Its as if the bigger community that I used to trust has been taken away. Further, the people you are taught to trust if you grow up as I did, the police, the authorities, can not/will not help you as a targeted person. They don’t even get the information about the case. One guy was saying they kept telling him it was a need to know basis. Well the civil defense folks/the government folks been instructed that I am a civil defense emergency or code to be dealt with or whatever. I wonder how many people who are targeted find themselves suddenly uncertain in trusting the old systems? If you are not a criminal, if you are not a terrorist, yet you are targeted and your job is renewed over and over again, you find trust to be a problem. The cumulative damage of having these terror protocols used on a person for years is awful. Its like running a gauntlet everyday you go out. How do you trust when people call you a terrorist? Can you trust the EMT to do his job if you are hurt? He is used to watch you. Can you trust the firefighter to rescue you if he thinks you are a terrorist? He’s in the civil defense network that calls you a terrorist. Can you trust the nurses in the hospital? They’ve been told you have links to the enemy, their husbands and family members might be in the military.

I think the trust issue will be interesting to read about. Best wishes Bruce!

Andre LePlume February 15, 2011 3:06 PM


Regarding the social contract and evolution, look at Brian Skyrms’ work. On non-kin altruism, if you have not already, be sure to read John Maynard Smith’s stuff.

AlanS February 15, 2011 3:28 PM

For angels we have knaves.

David Hume:
“Political writers have established it as a maxim, that, in contriving any system of government, and fixing the several checks and controls of the constitution, every man ought to be supposed a knave, and to have no other end, in all his actions, than private interest. By this interest we must govern him, and, by means of it, make him, notwithstanding his insatiable avarice and ambition, co-operate to public good. Without this, say they, we shall in vain boast of the advantages of any constitution, and shall find, in the end, that we have no security for our liberties or possessions, except the good-will of our rulers; that is, we shall have no security at all.”

Fredrik February 15, 2011 3:38 PM

“How does society — the honest majority — prevent the dishonest minority from taking over, or ruining society for everyone?”

It doesn’t! Have you not seen the endemic dishonesty in government and corporate leadership? It’s been this way for ages. Crooks rule the world!

Now, they are kept somewhat in check by the public, but only a little. For the most part, they do whatever they please, and far too few care enough to stop them.

Rajstennaj Barrabas February 15, 2011 3:53 PM

Dishonesty cannot be defined quite as readily as one might think. At best it is a spectrum with a wide range of values.

Dishonesty comes from the interplay between a rational agent’s self interest and the total utility to society. A surprising result from game theory is that there are actions that yield enormous total utility from the expense of an individual.

For example, consider Polio.

Unchecked, Polio will infect 10% of children, causing death and disability.

If 100% of children are vaccinated, no one will get Polio from the environment, but 1% will get polio from the vaccine.

Should a mother have her child vaccinated? A rational agent might reason that if all other children are vaccinated then they should avoid the 1% risk to their own child. They are essentially cheating the system.

(I’m using made-up numbers for clarity, but Polio really works like this.)

It is this interplay between the benefits/risks to the individual weighed against the benefits/risks to society that is the source of what we call dishonesty. Add in the risk/reward calculation and you have the motivations for just about anyone in any situation.

If the company you work for is doing unethical things, do you continue working for them? Even though you are contributing to unethical behaviour which hurts society overall, you can justify it by several means:

1) I’m only one of many who contribute
2) Other companies are unethical too
3) My boss makes the decisions
4) I’m getting paid really well for it

Points 1, 2, and 3 implement “fractional blame”, which reduces the risk. Point 4 increases reward. Now do the risk/reward calculation.

It’s easy to condemn child labour practices when the choice comes at no personal cost. But when presented with 2 brands of shoes, one of which is $100 cheaper, most people will go for the individual utility.

It’s a simple economic equation.

And finally, there is intelligence. One definition of intelligence is the ability to put off short-term gain for a long-term benefit. Thinking ahead in chess for example – the more moves ahead you can plan, the better player you are.

Companies can reduce costs by outsourcing production to Asia, where labor is cheap, and selling the goods here. But if every company does this then there will be no local jobs and no one will be able to afford anything. The companies will fail.

Angels are the ones who can think through the issues and believe in the total utility to society, even though their actions do not benefit them personally. They realize that their small personal sacrifices are vastly overcompensated by the benefits they receive from the small sacrifices made by everyone else.

The phrase “think and believe” above is key: Angels are smart enough to think through the issues, and they have the courage of their convictions.

We should all be like the Angels.

Jeremy February 15, 2011 4:20 PM

@Clive Robinson: “You first have to explain that there can be no good without evil and vis versa that is evil is the measure of good and good the measure of evil, neither can exist without each other. Therefore no Angels without fallen Angels.”

Regardless of opinions on the nature of good and evil (and there are many), in the context of this discussion, “angel” means something like “person who perfectly follows all of their society’s rules”.

Arguing that beings satisfying that definition could exist, but only in hypothetical realities that ALSO include “fallen angels” is patently ludicrous. How people in that reality think about them may be dependent on comparisons, but whether or not they satisfy that specific definition is obviously unchanged by the existence or nonexistence of other beings.

Daniel Rothman February 15, 2011 4:33 PM

In differentiating between personal and social safety, you might want to keep in mind the differences in allocation of costs. In a “safe” society of trusting individuals, there is some degree of attrition. At a societal level, there is some benefit to having a trusting society, offset at some point by the damage done by the dishonest. Part of what makes this fascinating is that each member of the group also has a sense of personal security, and in many cases being wrong about personal security is fatal.

So… there’s some equilibrium between the benefits of social trust, and the probability of personal annhilation.

Somewhere right in here there’s also room to talk about the selfish gene….

Dirk Praet February 15, 2011 5:33 PM

Wow ! Can’t wait to read the book.

But what are the criteria for “dishonest” ? What is dishonest to one individual or group, can be perfectly acceptable to another. Is dishonesty defined by rules and laws, by morality or by other, more transcendent concepts ?

My take is that once we get that cleared out, the goal of security should consist in minimising the benefits of dishonesty in both the short and the long term to the point that through education and experience we learn that nothing is to be gained from it.

Ryan February 15, 2011 5:36 PM

awesome! i’m looking forward to hearing more. best of luck with the research and early writing phase.

the question of why we naturally trust non-kin is an interesting one. i’m reading matt ridley’s “the rational optimist” right now – which i’m thoroughly enjoying so far – and he makes a pretty convincing argument that the reason might be trade. trade normally benefits both parties, and it also requires a base level of trust. from the evolutionary perspective, then, natural selection might favor non-kin trust because it’s needed for trade.

(of course, that only applies to species with brains that can conceive of trade, particularly barter style trade of unlike items, which ridley argues we’re uniquely capable of.)

Bruce Schneier February 15, 2011 6:25 PM

“But what are the criteria for ‘dishonest’? What is dishonest to one individual or group, can be perfectly acceptable to another. Is dishonesty defined by rules and laws, by morality or by other, more transcendent concepts ?”

I’m using “dishonesty” as a shorthand for betraying the group. So, depending on the group, it can refer to the criminal or the former criminal who turns in his fellows.

The book will explain this better, of course.

Bruce Schneier February 15, 2011 6:26 PM

“You first have to explain that there can be no good without evil and vis versa that is evil is the measure of good and good the measure of evil, neither can exist without each other.”

I’m not dealing in moral absolutes, but — using the nomenclature from the Prisoner’s Dilemma there can be no defection unless you know what it means to cooperate.

Mike February 15, 2011 6:26 PM

Be sure to address the cross-cultural issues here. High levels of trust exist in the US, but some countries have exceptionally low levels of trust.

Bruce Schneier February 15, 2011 6:28 PM

“Another form of society security a company employs is forcing employees to take their vacation. In fact, all such Internal Audits are a form of societal security, no?”

I see how internal audits are a form of societal security: the individual has an incentive to defraud his employer, and all the employees together have an incentive to minimize such fraud. But I don’t see how that works with mandatory vacation. Isn’t that just to get the unused-vacation liability off the books? Or are you thinking of banks, where forcing vacation time is a way for superiors to uncover employee fraud?

Robert Smart February 15, 2011 6:28 PM

Strongly recommend you have a look at some Frans de Waal books such as “Peacemaking among primates”. Note that some claim that humans are inclined to punish non-cooperaters, more than to cooperate. Arguably this mechanism breaks down in cities and networks.

Bruce Schneier February 15, 2011 6:30 PM

“Sociologists also have much to say on these topics.”

Suggestions of people to read would be appreciated.

Actually, that’s true for every discipline. A lot of the social sciences have much to say on these topics. And I would appreciate suggestions of people and papers and books to read.

Bruce Schneier February 15, 2011 6:33 PM

“We used to be able to kill off those that threatened the group. Society says that is not acceptable anymore – hence we pay dearly and there is no reason anymore not to be dishonest. The punishment for being dishonest, even blatantly dishonest, is increasingly mild in comparison with the damage is causes to the group.”

This is one of the important things I need to write about. As the amount of damage the dishonest minority can do increases, society can absorb an ever-decreasing number of them. And as the punishments for being part of the dishonest minority lessen, there is an ever-increasing number of them. It is unclear if there is a sable endpoint for this.

Bruce Schneier February 15, 2011 6:34 PM

“You will never find this trust in developing countries. Afrikan, Asian, Russian, Persian people never trust strangers. If you smile to a stranger in Moscow, you’ll easily get yourself into trouble. If you ask for help, and you don’t look like a person who needs any help, you won’t get any help from strangers.”

I agree with this; societal norms matter a lot. And I am going to talk about them.

But if you compare the human range of trust with that of other primate species, the trust we show to strangers is simply amazing.

Bruce Schneier February 15, 2011 6:37 PM

“But what are the criteria for “dishonest”? What is dishonest to one individual or group, can be perfectly acceptable to another. Is dishonesty defined by rules and laws, by morality or by other, more transcendent concepts?”

Since this is a book about security and not about morality, I am defining it in terms of the group. So a member of the Mafia who testifies against his fellows is “dishonest” from the point of view of that group. And, in fact, the primary security measure the Mafia uses to enforce honesty among its members is retribution.

C Good February 15, 2011 7:55 PM

I recently resubscribed to your Crypto-gram newsletter after a haitus of a few years. I am glad I resubscribed and look forward to your forthcoming book.

LandruBek February 15, 2011 7:58 PM

My squawk about the tentative title is just that the word “minority” has significant irrelevant associations — buttons that one doesn’t want to push. I’m sure a better title will occur to you.

To paraphrase Futurama: “I don’t know, I don’t know. Just use anything. As long as it’s compelling! mesmerizing! a tour de force!”

Andre LePlume February 15, 2011 8:12 PM


How do you operationalize “trust” for non-humans? Seems to me that for many species it is not at all rare. Are you simply saying that for non-humans life is nasty, brutish, and short?

Andrew February 15, 2011 8:37 PM

This former social ecologist looks forward to reading the book. What you are talking about seems to have been kicked around in the sociology of deviance as “formal and informal social control.” A riff on this, mentioning high trust and low trust societies in particular, is in John Ringo’s fiction book “The Last Centurion” which is available online (search “Baen CD Last Centurion” on your search engine of choice.) He distinguishes between “general trust” societies such as America and “familial trust” societies elsewhere in the world.

Betrayal of family for the abstract society isn’t even a question in much of the world — of course one puts one’s brother over abstractions of law and justice. The exception, that in this limited place and time, we have managed to privilege societal trust so much that it overrides kinship bonds — that is worth paying attention to.

Arvind Narayanan February 15, 2011 11:31 PM

I disagree on the necessity of human-like intelligence for trust in strangers. An inheritance-based mechanism is perfectly capable of evolving such trust over several generations, and in fact if memory serves Dawkins discusses just such a process in The Selfish Gene.

Clive Robinson February 15, 2011 11:38 PM

@ Jeremy,

“Arguing that beings satisfying that definition could exist, but only in hypothetical realities that ALSO include “fallen angels” is patently ludicrous”

If you go read what Bruce wrote he mentions fallen angels when talking about the Police and Armies so I did not introduce either concept.

However I think you compleatly missed my point.

How do you define good or evil? or anything else for that matter such as big or small.

To do it you need a “comparative measure”, that is you pick an example and make a comparison if something else is better or worse by perception.

After you do this a few times you develop a scale of measures such that you can say “how good” or “how bad” or “how big” or “how small” something is.

Measurment in all it’s various forms is one of the ways humans perceive the world and a fundemental step of science.

Thus I care little how you or anybody else defines a “good angel” or for that matter a “bad angel” as long as the difference is rationaly quantifiable to allow it to be usable as a method of measurment and thus used in prediction and test.

And that is my point to know “how good” or “how bad” needs a measure and refrence point.

Thus you cannot give rational meaning to “big/good/better” without having “small/bad/worse” to measure against.

Wayne February 15, 2011 11:49 PM

I am troubled with equating “society” with the “honest majority”.

It is entirely possible for dishonest people to get themselves into positions of power and authority.

When that happens, “society” becomes dishonest.

I’ve seen a number of research studies showing that the majority of students now cheat in school. I’ve also seen research that empathy is declining rapidly, and narcissism increasing rapidly, in American society, which came as a huge surprise to the psychologists who did the research because previously it had been believed that empathy and narcissism were determined primarily by genetics. Let me have a look on Google.

“Cheating Is On The Rise”

“Is The ‘Me Generation’ Less Empathetic?” “Young people today, compared to college students in the late 1970’s are 40% lower in empathy than their counterparts of 20 or 30 years ago.”

“Narcissism Epidemic Spreads Among College Students” “Data shows a significant increase in narcissism among Americans in the last 15 years.”

The reason for these changes is not a mystery. The people at the top of our society are rich people who are using all their wealth and power to get their hands on as much MORE wealth and power as possible. Furthermore they are lying to all the rest of us and deceiving us and telling us they are wonderful people who care so much about the common people and just want to make the world a better place. In this environment, empathy is a maladaptive trait — if you spend your money, your time, effort, and energy helping “losers”, what is going to happen to YOU in this vicious, cutthroat environment? You are going to end up homeless and hungry, aren’t you? Similarly, narcissism — this one is a little harder to understand, but I have noticed that relentless self-promotion is an asset in this environment. Humble people who are ineffective at marketing themselves get marginalized. And cheating, well, the advantage of that goes without saying — as long as you don’t get caught. And the cool thing is, the rich & powerful can mostly avoid getting caught, and can often get the rules changed so that their cheating isn’t “cheating” any more — i.e. how our laws today are written by lobbyists (see ).

I am not at liberty to describe my personal experiences, so I have to describe what I know indirectly through scientific studies; hopefully these links will give you some idea.

w February 16, 2011 12:06 AM

Can’t remeber were I heard this but opperssiion(group) and anangy(indvaual) bonce back and forth ever 400 years or so(not 100% sure).
Meassure the trust at those junctions might help with something

Sorry about my english

Richard Steven Hack February 16, 2011 1:17 AM

Wayne: That was precisely my point in my post – that investment in coercion producing observable monopoly profit results in more investment in coercion.

Unlike what most people believe intuitively, the state does not constrain coercion – it increases it. Because in reality there is no such thing as a “monopoly in coercion”. Anyone can coerce. If it is seen as profitable, it will increase until, as in all investments, the “general rate of return” is reduced to a very small number, which in social terms means the society is dysfunctional and a zero-sum game for everyone including the coercers.

And this is precisely the society we see around us, both in the US and in international relations. And the more the state increases its power, the more coercive it becomes and the more people invest in coercion to compete.

This continues until the state oppression and social coercion causes a completely dysfunctional society, at which point a revolution occurs, and things recalibrate to a lower level of state and social coercion.

The problem is that this recalibration never results in NO state because of the ten thousand year old intuitive belief system that the state is a “given” and the effect of primate hierarchy behavior which demands humans have leaders and followers.

Nothing except technology that alters human nature can change this dynamic. Fortunately over the next fifty years or more, this technology will exist, human nature will be altered, and the state and society that result from this dynamic will be eliminated. Probably along with a lot of humans who don’t adapt.

Lurker February 16, 2011 2:17 AM

“But I don’t see how that works with mandatory vacation. Isn’t that just to get the unused-vacation liability off the books? Or are you thinking of banks, where forcing vacation time is a way for superiors to uncover employee fraud?”

In some places yes.
However, it’s a practice regularly used to detect fraudulent activity.
The idea being if someone is concealing their routine fraudulent activities on a daily basis then someone replacing them for an extended period of time are likely to find out.
From what I understand it works pretty well, can be hard to enforce though (where I work had to stop the practice recently as it was decided to be against the/an employment agreement).

BrianSJ February 16, 2011 2:40 AM

John Robb has had some good posts on tribes e.g.

‘Modern’ society has developed an agency problem. People appointed to deal with societal security have developed their own agenda (regulatory capture needs to be more than an economic term, for example). The privileged nature of their position means that they are not currently amenable to collectivist web-based solutions.

Do finish the book, it is a really important one.

someone February 16, 2011 5:06 AM

Bruce, I guess you ignored/forgot another category… for example if I discover a new exploit, that happens because I was just curious about the vendor’s elaborateness.
I have no listed motivation.

Bruce Schneier February 16, 2011 8:40 AM

“Bruce, I guess you ignored/forgot another category… for example if I discover a new exploit, that happens because I was just curious about the vendor’s elaborateness.
I have no listed motivation.”

I don’t understand this comment. What is this another category of?

mcb February 16, 2011 8:51 AM


“So a member of the Mafia who testifies against his fellows is ‘dishonest’ from the point of view of that group.”

A gang member is also an untrustworthy member of the larger society. When he rats out his crew he takes a step toward returning to “trustiness” in the eyes of society. Are you going to examine issues related to the restoration of violated trust?

Mike W February 16, 2011 9:12 AM

There’s another cost of the increase in systematized law as society grows too large for universal personal relationships. Some things that may be normal or forgivable in Alice and Bob’s small village (like Alice filling her bowl with scrapings Bob tossed in his bin) can fall outside the bounds of legality. We can only imperfectly codify the behaviors we need to protect ourselves from.

Nevada February 16, 2011 9:37 AM

“I don’t understand this comment. What is this another category of?”

It looks like that was intended for the “Seven Types of Hackers” post above.

Nevada February 16, 2011 10:16 AM

“A gang member is also an untrustworthy member of the larger society. When he rats out his crew he takes a step toward returning to “trustiness” in the eyes of society.”

Not necessarily. In very early societies (to perhaps oversimplify things a bit) a PD decision might exist solely or primarily in relation to the tribe’s recognized authority and immediate welfare. Societies since then have been increasingly multivalent on larger scales: our decisions may be purely selfish and/or they may benefit any number of overlapping groups. A gang member’s defection might serve the larger society but it might easily be intended to serve any number of other gangs.

As Bruce has indicated a major theme and thrust of the research will be how this functions in highly-networked societies in which people may form sympathies and trust-bonds with multiple disparate groups with no geographical, ethnic, or cultural proximity.

One might also investigate what structural counterbalances there could be against this leading to extremes (total-surveillance states and cryptoanarchy, say, in which honesty- and trust-relationships would theoretically change radically).

Petréa Mitchell February 16, 2011 11:19 AM

I dispute the assertion that punishments have gotten lighter over time. Many countries still use the death penalty quite freely, and there are many things which are considered crimes today that wouldn’t have gotten a second look in any past era.

And we have entirely new categories of crime. Corruption in particular is something which requires a certain level of societal complexity to even exist.

Petréa Mitchell February 16, 2011 11:28 AM

“The first sentence should be corrected as ‘Western people have a natural propensity to trust non-kin, even strangers.’ ”

No it shouldn’t. Barring brain damage, every human being has the same capacity to trust non-kin. And every human being will manifest different levels of trust in different situations depending on their past experiences, their perception of the situation, and their personal sympathies.

“If you smile to a stranger in Moscow, you’ll easily get yourself into trouble.”

You can easily get yourself into trouble in New York City by looking too friendly, too.

“Unlike in Western countries, there are no self-checkout kiosks in supermarkets in poor countries.”

Do you suppose this could have something to do with the cost of technology?

“There are no book stands near the book shops on the streets, and no unprotected back doors in the private houses. Etc, etc.”

Neither of these are common in the rich Western country where I live, either.

mcb February 16, 2011 11:46 AM

@ Nevada

“‘A gang member is also an untrustworthy member of the larger society. When he rats out his crew he takes a step toward returning to ‘trustiness’ in the eyes of society.’

Not necessarily.

A gang member’s defection might serve the larger society but it might easily be intended to serve any number of other gangs.”

When I said “rats out his crew” I meant it in the positive connotation of becoming a rat fink, snitch, stool pigeon, or narc, rather than a disreputable lateral defection to another criminal enterprise.

Petréa Mitchell February 16, 2011 12:02 PM

Um, so one more nitpick. I’d also like to argue against making “dishonest minority” part of the title. As you noted in your post, most people (practically everyone) are dishonest from time to time, and the systems you want to talk about are designed to deal with everyday transgressions, not just the work of a few incorrigible evil geniuses.

And many transgressions are committed by people who break rules and can still see themselves as good and honest because the rule is nonsensical to them, or because it’s been overridden by local norms (“Everyone was doing it”), or because of conflicting rules (“I was just following orders”), or even because the action which breaks the rule is seen as prosocial (e.g., you say a flagrant instance of favoritism and violation of hiring guidelines; I say throwing a lifeline to my friend’s kid who’s on his last chance to go straight).

Alan Karp February 16, 2011 1:00 PM

I was the person at your RSA talk who asked if society needed the dishonest majority, perhaps to survive an attack from the outside. I didn’t mention it in the interest of time, but the movie “The Invention of Lying” has some insights to this problem that might be relevant. There’s no attack from the outside in that movie, but you can imagine the consequences.

paul February 16, 2011 1:36 PM

Perhaps the tendency to trust non-kin is a matter of typical universe size? My impression from primate researchers is that other primates typically exist in groups of tens to hundreds, whereas humans have for a long time existed in groups of hundreds to thousands. Not trusting (or not being trusted) is fairly expensive (including getting killed in the worst case), and at some number of contacts per year the odds of getting killed or severely compromised by not trusting must reach a turnover point.

Meanwhile, in a modern world the number of people a dishonest person can victimize keeps increasing. It started with mechanized transportation, where grifters could move to the next state, and so forth.

(Oh, and another voice agreeing that “The Dishonest Minority” is amenable to misparsing. Way too easy for people to misread and think there’s some pre-existing minority to whom the adjective should apply. Maybe more like “a few bad apples”.)

Dave B February 16, 2011 4:28 PM

Hi Bruce,

I sat in on the lecture at RSA this morning and the concept for the book is fascinating. Along a similar note I heard a report on NPR some months back titled “Is religion evolutionarily advantageous?”
The gist is is that all societies throughout the world have evolved the concept of a “higher being”, which begs the question, why have we evolved this way and what advantage is it to us evolutionarily.
I’ll let you read the rest. There’s a lot here relating to many of your ideas regarding the social contract.

The transcript is below. Let me know if any use.

Draft February 16, 2011 5:24 PM


I hope you’ll manage to address how a security system should work when the “dishonest minority [that] are the free riders of societal systems” are the people who run the societal system(s). That is, what happens when you have free riders who are at the top rather than the bottom. Say, for instance, military contractors doing horrible work and getting renewed contracts, or corporations that write loopholes and tax giveaways into laws, or banks that even after being bailed out are unreformed because they’re too big to fail.

I know that discussing those issues would require a broader definition of “security” but in at least the U.S. today the free riders are more at the top than the bottom.

Ross February 16, 2011 6:00 PM

One of the first things that jumped out at me involved the Alice/Bob bowl example. Other societies definitely have other ways to deal with cases where Bob may envy the bowl. I remember reading about cultures where if Bob admired the bowl enough Alice would be obligated (or feel some social compulsion) to give it to him, and there are other cases where those on top of society maintain their position by giving to everyone else to demonstrate their worth. In either case Bob might get the bowl he wants, without stealing even being considered. This might not directly relate to security as you’re exploring it, but it’s a striking contrast to American culture, and anthropologically interesting.

And for those squawking about the use of “minority” — don’t listen to them. It’s perfectly valid to use it in non-racial contexts. Looking at anything beyond the title should make things perfectly clear to any readers. And besides, if a few people briefly think the title is a hot-button topic, it’s just going to encourage them to look closer.

pc February 16, 2011 6:31 PM

Interesting topic for a book. I’d be one of the first in line to read it when it comes out.

It would be really interesting if you get a chance to explore the biological counterpart to this problem. Animal and plant evolution has dealt with this problem of the dishonest minority, of over millions of years by evolving various biological security measures. It might be interesting for you to compare and contrast the various biological and social security threats/responses.

One book which covers this topic in a decent amount of details is Richard Dawkins’ Selfish Gene (1976).

My 2 cents.

Tony February 16, 2011 7:16 PM

I was lucky enough to attend your 10:00 talk on “The Dishonest Minority: Security as Society’s Enabler.” Thanks for the insights and perspective. You asked for other readings on the subjects – one that I thought of was the Berger and Luckmann work “The Social Contstruction of Reality.” The book is a bit dated, but the research is very solid. Thanks, again.

John David Galt February 16, 2011 8:50 PM

This is very much a topic that deserves serious study, but I hope you won’t make the mistake certain political groups do of conflating government with “us”. Government is instituted to protect individuals, but once in place its members can be every bit as untrustworthy as their opponents — and frequently more so, both because governments frequently make their people immune to prosecution or lawsuit (or give them a monopoly on the right to prosecute) and because positions of power naturally attract the kinds of people who want to use them to bully or rob other people with the law’s blessing.

This is not to say that government is not worth having: only that “who will watch the guardians?” is a problem that never goes away, because governments of any kind — even those like ours, which was designed with permanent limits on its powers — always adapt to defeat any such limits and any mechanism meant to enforce those limits.

AC2 February 16, 2011 11:37 PM


‘The first sentence should be corrected as “Western people have a natural propensity to trust non-kin, even strangers.”
You will never find this trust in developing countries. Afrikan, Asian, Russian, Persian people never trust strangers.’

A striking example of rubbish, generalisation and a hint of racism in one compact package…

dmc February 17, 2011 8:38 AM

From Thomas Paine, Common Sense:

“Thus necessity, like a gravitating power, would soon form our newly arrived emigrants into society, the reciprocal blessings of which, would supersede, and render the obligations of law and government unnecessary while they remained perfectly just to each other; but as nothing but heaven is impregnable to vice, it will unavoidably happen, that in proportion as they surmount the first difficulties of emigration, which bound them together in a common cause, they will begin to relax in their duty and attachment to each other; and this remissness, will point out the necessity, of establishing some form of government to supply the defect of moral virtue.”

Chris Meissner February 17, 2011 4:08 PM

Hi Bruce –

Comment from you RSA talk. The lesson learned from the fact that technology enables a dangerous minority to be more effective is not that we need less minority to blunt the effect. Quite the opposite, it means that fewer an fewer minorities are necessary for greater impact. Although off the beaten trail for your subject matter I would peruse the Black Swan (not the movie alas) for perspective on this kind of dynamic. Would love go chat more on this topic as it’s one I’ve thought on a bit

Garrett February 17, 2011 7:24 PM

Humble suggested topic for the book …

Include a discussion on personality traits (and disorders, like psychopathy) and their societal effect. My own research in this leads me to believe that the millions of deaths committed by Stalin, Mao, Pol Pot, Hitler, et al, is due to a confluence of personality characteristics as well as fortuitous circumstances. I.e., lack of empathy, impulsivity, manipulativeness, and narcissism, as well as chance, led these men to positions capable of mass murder.

In societies where such people can’t rule over us, they can still do harm. The leaders–or misleaders–formerly atop WorldCom and Enron provide examples; a more recent example can be found in Bernard Madoff.

Machiavellian con men do societal harm and break the fabric of trust that holds societies together. It would be interesting to read your position on this from your security vantage point.

PackagedBlue February 17, 2011 10:38 PM

Sure hope such a book will be pragmatic for the next 10-15 years.

Sure hope it deals with the inflection change/paradigm shift, from BEFORE the end of the cold war.

Sure hope, it …. Well, nevermind… Then again, OpenSS L is like sendmail.

Otherwise, why bother, such security, is only, regulatory capturing.

Best regards to the book, and others hard work in psychology and societal security, however, in such “z” days, why would I expect anything real and useful today reported to the public?

Pete Simpson February 18, 2011 3:12 PM


Your definition: “Societal security is different. At the tactical level, it also involves attacks, countermeasures, and entire security systems. But instead of A vs. B, or even Group A vs. Group B, it’s Group A vs. members of Group A. It’s security for individuals within a group from members of that group. It’s how Group A protects itself from the dishonest minority within Group A. And it’s where security really gets interesting.” identifies not only where security becomes interesting, it gets to the core of arguably the most pressing problem facing our species at this evolutionary juncture.

Within Group A (mankind) we can define many subsets, but the subset of most interest in the context of societal security is that of the psychopaths. Informed estimates place this group at around 5 percent of the population.

An inability to empathize defines the psychopath: they are completely impervious to the suffering of others. They may lie, cheat, even commit acts of extreme violence unimpeded by any feelings of guilt, conscience or remorse. Given this competitive advantage they excel particularly in the fields of big business and politics. I can’t recall the source but remember that a senior FBI official went on record to the effect that the Bureau’s profile of a psychopath was an appropriate fit for the typical US politician. I recommend to you the works of the pre-eminent academic on matters of psychopathy, Robert D. Hare. In “Snakes in Suits: When Psychopaths Go to Work” Hare describes the condition and its implications for society very clearly.

Published research into any genetic basis for psychopathy seems lacking, but we can infer from genetic studies of its converse – altruism – that it may well be linked closely to the the AVPR2 gene, responsible for expression of arginine vasopressin in the the brain. Why is any genetic link of interest? Well, it just so happens this has a direct bearing on the operation of psychopaths as a discrete (not randomly dispersed) subset of Group A. Close interbreeding (cousins not siblings) such as has been practiced for centuries by for example the European Royal families and those dynasties that own the world’s central banks e.g.the US Federal Reserve, the Bank of England etc. has ensured (a) that they operate as an effective subset (kinship) and (b) retain a high level of psychopathy within their restricted gene pool.

At this time the subset, shall we call it subgroup P for psychopath or parasite, controls the process of creation of money. Governments must go cap-in-hand to borrow their own currencies from the privately-owned central banks. Subgroup P controls the mass media, international corporations, academia, governments and most importantly their national security apparatus. It is indeed a very sad state of affairs, but to investigate and report such matters invariably attracts the pejorative label: ‘conspiracy theorist’.

The reason why this constitutes a most pressing problem can be summed up in a word – interest. Not only must governments borrow their own money at source, they must also pay interest on the loan. It costs nothing for the central banks to create fiat currency out of thin air, but the conjuring trick does not include creation of the interest element. Thus we are consigned to a downward spiral of national and often personal debt. Ellen Brown articulates this well in “Web of Debt”. Economic and societal collapse are the inevitable end-results of interest-bearing fiat money creation. Quite simply, the parasite is in danger of destroying the host Group A. I sincerely hope you investigate these aspects of Societal Security but be warned of much muddying of the waters: disinformation and propaganda abound. And if you succeed in understanding the problem, conveying that to a wider readership faces the considerable obstacle of cognitive dissonance. I look forward eagerly to the fruits of your work.

Doug Coulter February 18, 2011 7:49 PM

I’ll pick a couple of nits, in what I hope is a helpful way. Even back in the days of hunter-gatherers security was quite expensive. Think the next tribe over comes and kills all your men, takes all the women and food. It’s never been cheap, and the job of sentry is always available.

The second best security (or army, or lawyer) is always even more expensive than the best.

Further, in some sense we’ve already lost the battle for societal security. We let people have power who lust after it so much they spend their lives learning how to get into power, then spending all their time trying to keep it instead of doing the job we “hired” them to do. Here we call them elected officials, and sometimes bureaucrats — who we don’t get to vote in or out (and are devilishly hard to fire or lay off too), but who can pretty much destroy a person or group with no apparent consequences to themselves.
And do so regularly.

And there’s this disparity of wealth thing, which I happen to be on the “lucky” side of — but then I earned it writing software and designing hardware, and kept it by not being a spendthrift.

Does a CEO these days deserve a multi million dollar bonus every year? Did they really create that much value that no one else could have? I’ve been a CEO of middling sized companies and was rewarded much more appropriately — never millions, even though the companies could have afforded that from things resulting from my work.
It simply never entered anyone’s mind! Does making one big financial deal really create so much value that the commission for the facilitator is worth that much? The few cases I’m aware of (say, Steve Jobs making digital music a reality by getting all the content providers to make a deal) — they don’t get the big numbers! But random stock traders working for big banks do.

Since money is power, more or less….

We’ve let these types control the world. We’ve lost.

Here in the US, we have a two party monopoly on power and it’s ever harder to tell which is which, but if you vote out one set of bums, the other set goes on calling it a mandate for themselves. No, you were just the only other so-called “choice” — we lose every single time.

Did you know congress critters have written a law that says it’s OK for them to do insider trading based on privileged information they can demand from companies or otherwise get in the course of what they do? Or vote on issues that will enhance their stock portfolios? That’s just one tiny example — there are many, many more.

I hope some of your words on these concepts find their way into your book — I’ll be buying it, anyway.

Good luck trying to cover such a complex subject, I have high expectations for the resulting work!

Jan Doggen February 19, 2011 4:15 AM

Bruce, at the completely opposite end of the spectrum (but it popped up in my mind because you were referring to implicit trust): you might want to read “The Gift” by Lewis Hyde. For the larger part it addresses the role of gifts in society (then moves to ‘art as a gift’, but that’s out of this context).
There are some thoughts in there that might add to what you’re doing.

John February 19, 2011 1:58 PM

A couple of people have posted comments about trust levels being different in different societies. I suspect this difference derives from the level and efficacy of societal security in the individual societies.

Another dimension that is important to consider is the spectrum from individualistic societies (like the USA) through to collectivistic (like Japan). The prevailing mental model of “who I am” in each society has a profound effect on what seems natural or not, and hence what has to be explained (or not).

Julien Couvreur February 20, 2011 11:07 AM

Seems interesting and a good continuation of your approach in using tools of economic analysis to security.

A few comments:
You will want to be clearer on what societal security means. You introduce a distinction with normal security, but to me they seem the same thing. All “societal security” threats come down to “normal security” threats. If the distinction is micro- versus macro-, then my recommendation is to stick with micro- as only individuals act and make decisions. Even group decisions are traceable down to individual decisions.

Secondly, you are obviously correct that as the conditions change (population density, technology, costs), then the threats evolve and the defenses have to adapt. But the information age does not change the analysis, it is not a paradigm shift, the same logic human action still applies, only the trade-offs change.

Thirdly, I would suggest not to jump too quickly from the existing of a problem (some people cheat) to the necessity of government. I’ll give some pointers below, but the basic idea is that monopolies are rarely the best solution, and monopolies on security services are no exception. On top of that, the fact that unlike normal monopolies government has a monopoly on aggression including the ability to tax makes it an even worse idea. When it comes to “societal security”, I would count government in the camp of threats, just like a mafia. At the very least, it is not the cheapest way to solve societal security problems.

Finally, I would question the value of discussing the life of angels. It is easy to fall into a nirvana fallacy.
Even in small and very trusting communities, currency still existed. NPR mentioned the big rock wheel currency last month. People didn’t physically move it when it changed owner. They trusted to know who’s the current owner. They even chose to accept a unit of currency that had sunk in the sea. The point is that money and currency would still exist, but the burden of proof would be much cheaper or zero (people trust what you say).

Julien Couvreur February 20, 2011 11:13 AM

“Government (or some equivalent) would still be necessary because even given perfect people they still have limited knowledge.”

Yes, knowledge is costly to aggregate (for example, the census). But it does not follow that government is necessary, as defined as a territorial monopoly on force. Instead, those problems can be solved voluntarily thru market solutions. If the census is really valuable, why can it only be funded thru involuntary contributions (taxes)?

Julien Couvreur February 20, 2011 11:35 AM

Regarding the so-called “tragedy of the commons”: this is often due to a difficulty or a failure to define property.
Owners have incentives to sustain, develop, as opposed to deplete. Some examples: overfishing in a lake, keeping streets clean from trash, pollution in a river, poaching of elephants/rhinos. All those can be traced back to property not being allowed, defined or enforced.

Carl Grayson February 21, 2011 1:57 PM

I think one of the difficulties we experience now (as opposed to in earlier times) is the increased population interaction density has caused us to make more and more explicitly defined “rules”.
The ability to be aware of (and balance) all societal rules that could be in effect at any time is not trivial for an undertaking of any serious size (try navigating law (especially across jurisdictions), industry regulation, and social opinion sometime).
One problem created by how we’ve tried to solve this is how brittle some these social rules are now becoming under failure, especially in the case of law (it didn’t work? make a new law / expand an existing one – and increase the original problem via feedback).

K February 22, 2011 12:43 PM

Kaczynski again.
There is a disease.
It is called ‘Security’.
It is infesting every corner of life.
It controls who and when we can fly.
It controls the data moving within electronics.
Almost every function in technology and society either has, or is developing, a security component.
This component not only regulates the use of resources, it consumes resources itself.
In some situations, it consumes more resources than the actual function which it is regulating.
It is a cancer choking the free flow of resources.
It is based on the premise that nothing and no-one can be trusted.
One day the race will wake up and find that ‘Security’ means no freedom. Whatsoever.

Martin Budden February 26, 2011 6:09 AM

I am always wary when I hear an assertion that a particular characteristic is unique to humans. Partly because I think it is arrogance, but mainly because it just does not make evolutionary sense. There are many assertions of unique human qualities, without really considering what this means in evolutionary terms: if these assertions are taken as true it implies that, in the last million years or so, a large number of characteristics have simultaneously evolved in a single species. I just don’t think this is very likely. I think it is a much better null hypothesis that incremental improvements in a number of areas have evolved in humans, and, because of the non-linear nature of ecosystems, these have given humans a disproportionate evolutionary advantage. I accept that humans may have evolved unique characteristics, but believe that such claims should be challenged until there is good evidence.

So back to your claim:

“Humans have a natural propensity to trust non-kin, even strangers … except for a few simplistic counterexamples, it’s unique among life on this planet.”

I disagree. I think trust of non-kin is extremely common.

Trust involves three things: an implicit or explicit deal or bargain, a belief in the competence of the other party, and a belief that the other party won’t cheat. In the absence a communication mechanism or common language (as is the case in most of the natural world) the bargain is, of course, implicit. A deal can be made without trust but it means both parties to the deal have to spend significant amounts of energy checking up on each other and enforcing the terms of the deal. Bringing trust into the equation means that both parties can spend less energy on checking and therefore more energy on more productive things. Continuous checking that the bargain is being upheld is replaced by taking some form of punitive action when the bargain is breached. The punitive action may be as simple as not dealing with the incompetent or cheating party again, or it may involve inflicting some form of harm on the party that breached the agreement.

Note also that trusting someone does not mean that you believe that they do not cheat, it just means you believe that they do not cheat above a level implicit in the deal. Some cheating is tolerated, since a deal with some cheating can be more beneficial to both parties than no deal at all.

If I had to boil down the essential manifestation of trust, I would say it is this: the replacement of continuous verification by intermittent checking and a punishment/reward mechanism.

So let’s get back to biology.

I’ll begin with a simple statement: symbiotic relationships are extremely common in biology. By my definition of trust, symbiosis is a trusting relationship between organism that are not members of the same species. Of course parasitic relationships are also extremely common in biology, but the existence of parasitic relationships does not deny the existence of symbiotic ones. If non-species trust is common, then non-kin trust within a species is also likely to be common.

I imagine that there are two scenarios where trust relationships are particularly prone to evolve:
i) groups of prey cooperating in predation defence
ii) groups of predators cooperating in the hunt

Whenever a group of animals larger than a kinship group forms, there is some kind of implicit deal made. It’s almost inevitable that trust evolves, since if the deal is not upheld a member of the group can sanction the group simply by leaving.

Here’s an example of monkeys cooperating (sharing food supply) because of predation defence benefits ( A single monkey could cheat – imagine it saw a bunch of fruit, but another monkey was closer – the cheating monkey could give the danger call and then get the fruit for itself. I imagine detection and sanction mechanisms have evolved to keep this behaviour to a tolerable level.

Although cooperation and trust within a group of prey is interesting, my view is that the cooperation and trust mechanisms within groups of predators are more complex and more interesting. There is just so much more going on. Assessment of competence is more important (is that young pup ready to join the hunt? is that injured group-member too much of a burden?). There is the issue of sharing out the kill – note that the criteria for fairness is not “are the shares equal”, but is “are the shares what was implicitly agreed before the hunt” – everyone expects the “top dog” to get more than an equal share, but even the “bottom dog” expects a small share. Predator groups are often large than familial units, so there must be non-kin trust within these groups.

As well and predator and prey groups, trust can be beneficial to a group with a common goal. An example that is particularly interesting to me is that of a flock of geese in a V-shaped skein. It’s particularly interesting to me because I am a cyclist, and cyclists often ride in “pacelines” where, like a skein, the effort of being on the front is shared. A paceline makes an interesting societal security case study. Pacelines form because it is mutually advantageous to all the cyclists involved – the energy expenditure in a paceline is (for riders not on the front) is about 30% lower than the energy expenditure of a rider on their own. Pacelines assume trust – both in competence and in willingness to do a fair share of time at the front. Mechanisms exist for ejecting riders who are incompetent (and/or dangerous) or who don’t pull their fair share. Mechanisms also exist for protecting club members or team members who are struggling with pace. It would be an interesting study to see if similar mechanisms exist in flocks of geese.

In your post you say “Security isn’t just a tax on the honest; it’s a very expensive tax on the honest”, ” Security — especially societal security — used to be cheap” and “we routinely trust non-kin (an evolutionary puzzle, to be sure)”. You first two statements answer the evolutionary puzzle – trust of non-kin evolved because societal security is cheaper than individual security. There is no evolutionary puzzle: trust (by which I mean the replacement of continuous monitoring by detect and punish) is the cheapest way to improve security. An animal that trusts others, even if it is occasionally cheated, is likely to be more successful than an animal that does not trust others. (Of course that trust cannot be simple naive trust – the animal also needs to be able to evaluate if another animal is trustworthy and if it is being cheated.)

Which leads we to a final observation about societal security. You say it used to be cheap, implying it is getting more expensive. Could this be because society is moving away from trust based security (detect and punish) to prevention based security (where we try and prevent wrongdoing by, for example, continual monitoring and surveillance)?

Nick Coghlan March 1, 2011 12:58 AM

I’ll echo those suggesting you may want to reconsider the title. If you’re using “dishonest” to mean “betraying the trust of a group you are part of”, rather than its conventional meaning then why not actually include the words “trust” and “betray” in the title? Perhaps something like “Trust and Betrayal: Security and its Role in Modern Society” or “Betrayals of Trust: Security and its Role in Modern Society” or even “Trust and Betrayal: Societal Security in a Modern World” (yes, the last is a deliberate riff on Secrets & Lies and its subscript).

As far as research material goes, a recommendation a friend of mine pointed out after reading your post is Matt Taibbi’s “Griftopia”, as well as his other articles on the disturbingly cosy relationship between Wall Street and the SEC (e.g.

Christine Boisvert July 16, 2011 1:50 PM

I have a quote from Charles Darwin;

A fair result can be obtained only by fully stating and balancing the facts and arguments on both sides of each question.

Sandy Harris July 24, 2011 10:34 PM

Your approach is excellent as an overall abstraction for why we need security, starting from human factors which is probably the right place to start.

A complementary view is possible, starting from the systems side, and it may add something.

The great linguist Sapir’s line “All grammars leak.” ( applies to much more than just grammar. Formal systems in general don’t do well at describing human behaviour. In particular, all AI problems are hard, but it is not just those.

A code of laws needs lawyers and judges to make it work, and there will always be sneaky-clever people who “get away with” things the lawmakers never thought of. Same for tax systems & accountants, computer systems & hackers (in both senses of that term), etc.

Robert December 15, 2011 12:37 PM

Idea: come up with some sort of cipher/puzzle, possibly several with different messages, and give away one copy per puzzle-solver.


Felipe Allende December 15, 2011 4:29 PM

possible idea/s
– provide somewhat of a hidden message puzzle
– have your readers provide some kind of steno-graph message and the best one wins
– have your readers provide the best security related story or experience
– provide a particular security scenario that requires the opinion from one of us and the best one wins…

great books and keep up the great work can’t wait to read your latest book

Jim Caron December 19, 2011 1:52 PM


Book? What book?

Use a “Drop/Mail-Box” like all retro-era spy-geeks.


P.S. Sorry for the late delay in commenting. I just got back from the land of ice & snow.

Leave a comment


Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via

Sidebar photo of Bruce Schneier by Joe MacInnis.