Bruce Schneier

# Schneier on Security

A blog covering security and security technology.

## April 7, 2011

### Detecting Cheaters

Our brains are specially designed to deal with cheating in social exchanges. The evolutionary psychology explanation is that we evolved brain heuristics for the social problems that our prehistoric ancestors had to deal with. Once humans became good at cheating, they then had to become good at detecting cheating -- otherwise, the social group would fall apart.

Perhaps the most vivid demonstration of this can be seen with variations on what's known as the Wason selection task, named after the psychologist who first studied it. Back in the 1960s, it was a test of logical reasoning; today, it's used more as a demonstration of evolutionary psychology. But before we get to the experiment, let's get into the mathematical background.

Propositional calculus is a system for deducing conclusions from true premises. It uses variables for statements because the logic works regardless of what the statements are. College courses on the subject are taught by either the mathematics or the philosophy department, and they're not generally considered to be easy classes. Two particular rules of inference are relevant here: modus ponens and modus tollens. Both allow you to reason from a statement of the form, "if P, then Q." (If Socrates was a man, then Socrates was mortal. If you are to eat dessert, then you must first eat your vegetables. If it is raining, then Gwendolyn had Crunchy Wunchies for breakfast. That sort of thing.) Modus ponens goes like this:

If P, then Q. P. Therefore, Q.

In other words, if you assume the conditional rule is true, and if you assume the antecedent of that rule is true, then the consequent is true. So,

If Socrates was a man, then Socrates was mortal. Socrates was a man. Therefore, Socrates was mortal.

Modus tollens is more complicated:

If P, then Q. Not Q. Therefore, not P.
If Socrates was a man, then Socrates was mortal. Socrates was not mortal. Therefore, Socrates was not a man.

This makes sense: if Socrates was not mortal, then he was a demigod or a stone statue or something.

Both are valid forms of logical reasoning. If you know "if P, then Q" and "P," then you know "Q." If you know "if P, then Q" and "not Q," then you know "not P." (The other two similar forms don't work. If you know "if P, then Q" and "Q," you don't know anything about "P." And if you know "if P, then Q" and "not P," then you don't know anything about "Q.")

If I explained this in front of an audience full of normal people, not mathematicians or philosophers, most of them would be lost. Unsurprisingly, they would have trouble either explaining the rules or using them properly. Just ask any grad student who has had to teach a formal logic class; people have trouble with this.

Consider the Wason selection task. Subjects are presented with four cards next to each other on a table. Each card represents a person, with each side listing some statement about that person. The subject is then given a general rule and asked which cards he would have to turn over to ensure that the four people satisfied that rule. For example, the general rule might be, "If a person travels to Boston, then he or she takes a plane." The four cards might correspond to travelers and have a destination on one side and a mode of transport on the other. On the side facing the subject, they read: "went to Boston," "went to New York," "took a plane," and "took a car." Formal logic states that the rule is violated if someone goes to Boston without taking a plane. Translating into propositional calculus, there's the general rule: if P, then Q. The four cards are "P," "not P," "Q," and "not Q." To verify that "if P, then Q" is a valid rule, you have to verify modus ponens by turning over the "P" card and making sure that the reverse says "Q." To verify modus tollens, you turn over the "not Q" card and make sure that the reverse doesn't say "P."

Shifting back to the example, you need to turn over the "went to Boston" card to make sure that person took a plane, and you need to turn over the "took a car" card to make sure that person didn't go to Boston. You don't -- as many people think -- need to turn over the "took a plane" card to see if it says "went to Boston" because you don't care. The person might have been flying to Boston, New York, San Francisco, or London. The rule only says that people going to Boston fly; it doesn't break the rule if someone flies elsewhere.

If you're confused, you aren't alone. When Wason first did this study, fewer than 10 percent of his subjects got it right. Others replicated the study and got similar results. The best result I've seen is "fewer than 25 percent." Training in formal logic doesn't seem to help very much. Neither does ensuring that the example is drawn from events and topics with which the subjects are familiar. People are just bad at the Wason selection task. They also tend to only take college logic classes upon requirement.

This isn't just another "math is hard" story. There's a point to this. The one variation of this task that people are surprisingly good at getting right is when the rule has to do with cheating and privilege. For example, change the four cards to children in a family -- "gets dessert," "doesn't get dessert," "ate vegetables," and "didn't eat vegetables" -- and change the rule to "If a child gets dessert, he or she ate his or her vegetables." Many people -- 65 to 80 percent -- get it right immediately. They turn over the "ate dessert" card, making sure the child ate his vegetables, and they turn over the "didn't eat vegetables" card, making sure the child didn't get dessert. Another way of saying this is that they turn over the "benefit received" card to make sure the cost was paid. And they turn over the "cost not paid" card to make sure no benefit was received. They look for cheaters.

The difference is startling. Subjects don't need formal logic training. They don't need math or philosophy. When asked to explain their reasoning, they say things like the answer "popped out at them."

Researchers, particularly evolutionary psychologists Leda Cosmides and John Tooby, have run this experiment with a variety of wordings and settings and on a variety of subjects: adults in the US, UK, Germany, Italy, France, and Hong Kong; Ecuadorian schoolchildren; and Shiriar tribesmen in Ecuador. The results are the same: people are bad at the Wason selection task, except when the wording involves cheating.

In the world of propositional calculus, there's absolutely no difference between a rule about traveling to Boston by plane and a rule about eating vegetables to get dessert. But in our brains, there's an enormous difference: the first is a arbitrary rule about the world, and the second is a rule of social exchange. It's of the form "If you take Benefit B, you must first satisfy Requirement R."

Our brains are optimized to detect cheaters in a social exchange. We're good at it. Even as children, we intuitively notice when someone gets a benefit he didn't pay the cost for. Those of us who grew up with a sibling have experienced how the one child not only knew that the other cheated, but felt compelled to announce it to the rest of the family. As adults, we might have learned that life isn't fair, but we still know who among our friends cheats in social exchanges. We know who doesn't pay his or her fair share of a group meal. At an airport, we might not notice the rule "If a plane is flying internationally, then it boards 15 minutes earlier than domestic flights." But we'll certainly notice who breaks the "If you board first, then you must be a first-class passenger" rule.

This essay was originally published in IEEE Security & Privacy, and is an excerpt from the draft of my new book.

EDITED TO ADDD (4/14): Another explanation of the Wason Selection Task, with a possible correlation with psychopathy.

Anon Y MussApril 7, 2011 1:40 PM

The first test seems senseless or arbitrary. The second test does not. Resolving the skeleton of a scenario, implied by the isolated constraints, into a satisfactory, concrete situation is what is failing in the first test.

I'm sure plenty of subjects wondered "Why is car travel restricted into Boston? For how long?", and I'm sure they're misinterpreting the "if, then" as an equivalence. If it were a real situation, much more information would be required to make a decision about the fact that all Boston travel is by plane.

The reason they're thinking that is that they aren't abstracting the situation; they're looking for causes, effects, and correlations that seem familiar to them. I disagree with characterizing this as "cheating" in a "social exchange", based on the examples given. It's a simple matter of concreteness.

In an essay published in the book Conceptual Issues in Evolutionary Biology, 3rd edition. (2006, Elliot Sober, ed.), David Buller argues that the Wason selection task as used in evolutionary psychology is flawed. According to Buller, we think that there is a content effect where the frequency with which individuals select the "right" answer is a function of what the conditionals *are about* - it's easier to select when it's about cheating because we're wired to see cheating. But this hinges on the assumption that the conditionals in both tasks (cheating vs. abstract) are of the same form, and Buller argues that they in fact aren't; as he states it, the abstract case is an indicative conditional, where the cheating case is a deontic conditional. If you're interested in this issue, and evolutionary psych in general, the essay is worth a read.

Brandioch ConnerApril 7, 2011 1:46 PM

@Anon Y Muss
Good point.

And easily tested. State the situation in terms of tuxedos, jeans, going to the opera and going to the movies.

Can the people still make the correct connections?

Or carrying a toolbox, carrying flowers, working on the car vs wife's birthday.

Or did I, like many, not understand the test?

Joe BuckApril 7, 2011 2:08 PM

I think that the problem is that the selection task as described gives an arbitrary rule that makes no sense, and asks people to apply it. The natural instinct in such cases is to try to correct the rule. If people applied purely formal logic, then as soon as a contradiction is successfully introduced into our knowledge base, we can deduce that any statement is true, and also that any statement is false! It would be all over for reason.

Consider a rule that says "if you're in the green room, you must have gone through the red door". There's no reason to see being in the room as a social benefit or going through the door as a social cost. But we can picture the situation: there's a room, and the only way in is through the door. Plop this rule down into a game of Clue and I think people would have no problem with the equivalent of the selection task, because people don't instinctively use propositional logic. What they do instinctively do is picture the situation in their head and work things out using their model of the situation. So if we say "Jane didn't go through the red door", people would have no trouble saying that she's not in the green room.

It might be interesting to do some randomized experiments where the property being checked is logically the same but expressed in several different ways:

1. True or false: If somebody buys flowers, then they are wearing a hat.

2. True or false: Everyone who buys flowers also wears a hat.

3. The king has decreed that everyone who wants to buy flowers must wear a hat. Is this decree being respected?

4. The king has decreed that only people who wear hats are allowed to buy flowers. Does anyone cheat?

If the hypothesis is correct, people should perform markedly better at question 4 than at question 1.

(One point would be to avoid contamination from people's ideas about whether the rules being tested are meaningful and/or desirable).

MatthewApril 7, 2011 2:13 PM

This makes sense that the wording would affect the result - we are self interested creatures. The vagaries of cost / benefit are more closely aligned with our sense of self interest than worrying about who took a plane to boston or what P or Q means.

We are attuned to catch cheating because we hope that when we pay a cost, we are entitled to our benefit. In some, the expectation may be that if there is a way to cheat the system, they will be able to find it and exploit it. For none other than self interest.

There seems to be an error:
"
If you know "if P, then Q" and "not Q," then you know "P."
"
- this seems pretty much impossible, if you know p->q and not q, that wound mean nothing about p.

Typo:
"if P, then Q" and "not Q," then you know "P."
should read "if P, then Q" and "not Q," then you know "not P."

@Typo, that's wrong.
"If p and only if P then Q" and "not Q", then "not P". Otherwise you know nothing, because something else might be true, like R -> Q.

@Vasil: It should be:

If you know "if P, then Q" and "not Q," then you know "not P"

But that does let you conclude something from p->q and not q, namely not p. Because either p or not p, but if p then (by assumption) also q, and we know (also by assumption) that this is not false. So the only remaining possibility is not p, which must therefore hold.

Argh. I meant ".. that this is not true".

EarwigApril 7, 2011 2:27 PM

But.

No, you don't understand the logic:
"if you know p->q and not q, that would mean nothing about p"

Quite to the contrary, it means everything about p:

If p's truth requires q, but q is known to be false, then p can't be true.

@Henning Makholm , nothing says that not q -> not p, e.g. there's no problem that P is false, but Q is still true.

Let me try with an example:
P - someone is human
Q - walks on two legs

So, if someone is human (P), he walks on two legs. But, if he doesn't walk on two legs, he might still be human. The link is ->, not the , or at least this is the way I understand it, "if x then y", not "if X and only if X, then Y".

Note also that the prohibition in formal logic against inverting Modus Tollens -- that is, going from
A=>B to B=>A, instead of to ~B=>~A -- is weakened in inference.

Inference, the passage from evidence to conclusions, is a generalization of logic, the passage from premises to implications. It is the language of science, and more generally of practical reasoning. As a set of formal logical statement, inference looks like this:

(1) A=>B1; A=>B2; A=>B3; ...; A=>BN
(2-1) B1 is true => some evidence exists for A;
(2-2) B2 is true => more evidence exists for A;
...
(2-N) BN is true => a lot of evidence supports A.

So, for example, while "took a plane" is not proof that someone "went to Boston", it is (weak) evidence that that person may have gone to Boston. In conjunction with the truth of a lot of other "non-proof" statements whose falsity could contradict "went to Boston" ("ate chowder", "heard about a Red Sox game", "now drives like a maniac"...), it can actually build a case that the person in fact went to Boston, without constituting actual proof.

The point is, in ordinary reasoning, logical proof is often useless. It is too paralizingly stringent a criterion for accepting a conclusion. Reasoning people infer conclusions from evidence far far more often than they get to prove a consequence from a premise. The technical details of Modus Tollens are irrelevant to most thinking, except insofar as someone who constantly goes from A=>B to B=>A is demonstrating an inability to distinguish weak evidence from strong evidence.

@Earwig, Q's truth requires P, there's nothing said for the reverse direction.

Dave KapellApril 7, 2011 2:37 PM

@Earwig: I've always used the following example:
"If it's raining, the grass is wet"

P = It's Raining, Q = The grass is wet
If we know the the grass is wet(Q), we don't know if it's raining(P), or if someone turned on the sprinkler (~P(or R, if you prefer)). If we know the grass is not wet (~Q), then we know that it's not raining (~P), or else the grass would be wet.

SpaceHoboApril 7, 2011 2:58 PM

Wow! This was like a damninteresting.com article, but without the hyperbole or embroidery. Bravo!

@Vasil:

"not Q --> not P" is the same statement as "P --> Q". It's called the contrapositive (http://en.wikipedia.org/wiki/Contraposition), and is one of the first things you learn in logic class.

Another possibility, along the lines of Joe Buck's comment, is that the brain is used to correcting its model, not assuming its inputs are faulty. We're not used to "given the model you have, which of these statements need confirmation," we are used to "given these confirmations, is your model consistent?"

With his example of the "If you are in a green room, you went through a red door," we are not used to asking if Jane could be in the room...she is, or she isn't. Rather we are used to seeing that she is indeed in the room, and working the logic from there.

MarcusApril 7, 2011 3:17 PM

Talk about food and everybody understands. I am sure you can phrase the same problem as poker or taxes and still have many people who don't get it.

DavidApril 7, 2011 3:40 PM

"Once humans became good at cheating, they then had to become good at detecting cheating -- otherwise, the social group would fall apart."

This is wrong - there is no need to posit group selection (a much weaker force than ordinary selection - groups don't die nearly as often as individuals). Given that other humans and social obligations were a big part of the environment of our ancestors, both detecting cheating and improved cheating would benefit the individual.

The tv classic Mary Tyler Moore had a scene where the bumbling news anchor Ted Baxter couldn't solve a simple arithmetic problem, until Lou Grant told him "Put a dollar sign in front of it, Ted", which allowed Ted to arrive instantly at the correct answer.

Alan GreenApril 7, 2011 4:51 PM

Fascinating. In my day job as a software engineer, my colleagues and I often have strong feelings about how problems are solved in code, and discussions can be quite heated, with much indignation all 'round. I wonder whether we are transferring some of our special morality recognition brain power to the task of writing code.

Dirk PraetApril 7, 2011 5:02 PM

The simple explanation is that as soon as cheating is involved, the problem on the table gets filtered through the subconscious part of our mind that decides on whether or not an action is right or wrong, based on a person's reference framework of what is just, and what isn't. It would be quite interesting to conduct the Wason selection task - with and without the "cheating" cases - on an audience of known sociopaths. I bet the results would be entirely different as those from "normal" people.

Rob FunkApril 7, 2011 5:03 PM

I suspect that the Boston/plane problem would suddenly become easier if it were changed to Hawaii/not-car. Which has nothing to do with cheating, and everything to do with arbitrariness.

In other words, "what Anon Y Muss said."

DavidApril 7, 2011 5:07 PM

@Vasil

The fact that you could walk on other than two legs and still be human (if you were, say, an infant or a paraplegic) does show that

not two legs -> not human

does not hold. It does not, however, show that the equivalence does not hold because the implication is invalid - not all humans walk on two legs. The "implication" is statistical, not logical.

Look at it this way:

Truth table for
P -> Q

P | Q | P -> Q
T | T | T
T | F | F
F | T | T
F | F | T

Truth table for
not Q -> not P

P | Q | not Q -> not P
T | T | T
T | F | F
F | T | T
F | F | T

As you can see, for the same P and Q, the two statements have the same truth value, and so they are equivalent.

Petréa MitchellApril 7, 2011 5:14 PM

Dirk Praet:

It's been done, and the sociopaths do indeed perform significantly worse on average than non-sociopaths.

@Brandioch Conner, would the answer be
toolbox -> not wifesbithday
carrying flowers -> not working on the car
?

pfoggApril 7, 2011 5:23 PM

@stvs

There was a formal experiment matching that, where children were asked basic arithmetic problems phrased simply as numbers, and as quantities of money. The children asked about the money did much, much better.

Interesting Article, thanks for posting it.

Clive RobinsonApril 8, 2011 1:35 AM

The problem arises because of multivalance and the individuals perception of the world around them.

Formal logic deals with bivalence that is in normal language 'something is either true or it is false'. That is it can not be a "little bit true" or a "little bit false" or a "bit of both" which is the general state of the world we percieve. That. is we say something is "black or white" but which as humans we actually know is false as a statment simply because we are aware of gray and colour.

Thus if you ask a human a question with a fundementaly bivalent answer such as "is the person cheating?" they will either reason it out (to yes or no) or say 'I don't know'.

But as humans we generaly don't ask questions in that way, we say such things as "Do you think that person is tall" which without a common refrence point cannot be meaningfully answered in a bivalent way.

But as humans we still generaly give a bivalent answer as yes or no if the person is clearly tall or clearly short by some personal perception point, but will say "a little tall" or "a little short" or "about average" if the hight is close to the personal perception point. That is we have shades of grey about an arbitary perception point.

Thus the first thing that needs to be examined is the quality of the question used for the test.

If the question is such it produces a naturaly bivalent response in a human then it is suitable for the test. But if the question can have a multivalent answer or does not naturaly produce a bivalent response then it is not suitable as a question.

If you think about it most things in life have multivarient answers, even "is it raining" has a mid point that causes vairied answers such as drizzle or misty. Or worse as in transportation have many possible answers (walk, run, cycle, drive, by bus, by train, by aircraft or even by boat).

The thing about "socrates" is his status as a "man" is accepted as bivalent and so is his mortality. We naturaly think of a person as being a "(hu)man or not" and likewise "alive or dead". But what if we changed the question and used sitting and awake instead?

Likewise we all say "heads or tails" for a coin toss, even though a few of us have seen a coin land on it's rim and stay there. We say "heads or tails" because that's the generaly held "natural order" or perception of it not the actuality or reality of it.

So as with so many of these "test the mind" experiments you have to be very sure your experiment is testing what you think it is.

As with "rats in mazes" and "upright nuns" it is very very easy to get the experiment wrong and thus come up with meaningless answers.

DanielApril 8, 2011 2:57 AM

Interesting post on similar subject @ the last psychiatrist... titled "Test Of Psychopathy?" but that's misleading. highly recommended reading imho.

http://thelastpsychiatrist.com/2010/12/...

GreenSquirrelApril 8, 2011 3:29 AM

@Anon Y Muss and @Rob Funk

The vegetables - dessert question is just as much an arbitrary relationship as flying to Boston.

However, as we (developed world at least) tend to be in the mindset that children dont want to eat vegetables so they have to be bribed by the offer of a dessert, it makes "more sense" to us as a set of questions.

I *think* the point here is that this is because: A) we are conditioned to see this as a cost/benefit scenario; B) we are conditioned to detect cheaters.

In reality, demanding that vegetables are eaten before a dessert can be granted is easily as arbitrary as saying you can only get to Boston by flying so the only genuine difference is one appears to involve potential cheats and the other doesnt.

(some possible bias points for me - the only way I can get to Boston USA is by flying and my children would rather eat their vegetables than the meat on their plates)

StaudenmaierApril 8, 2011 3:58 AM

VERY enjoyable, Chief!

A minor point I noticed is that the basic logic term "contrapositive" is strangely avoided, but that's merely preference.

Patrick G.April 8, 2011 4:39 AM

Maybe people just can't understand mathematicians and their love of overly abstract settings that tend to defy common sense.

Gianluca GhettiniApril 8, 2011 4:45 AM

I think the that the Boston/plane problem is so hard for people to understand because of the arbitrariness of the P->Q statement. Not all airplanes go to Boston so we get confused by the strict implication P->Q. As humans, we live in a world of unavoidable trade-offs and we are used to it. Rewrite the P->Q statement as "if it's heavy then it's difficult to move by hand". That sort of statement is unavoidably true in the real world and the problem suddenly become easy.

This article would be made much greater by a rewrite. Add examples of how the premise relates to social networks and republish. Thanks.

Gianluca GhettiniApril 8, 2011 7:29 AM

@Irene Irene
In other words, "what Anon Y Muss said."

again'n'again,
wtf are u spewing out?

It's actually very simple to comprehend the logic and equally simple to explain why Wason tests have a low success rate. Nobody cares who went to Boston or how, but if I only get dessert for eating my veggies, then I'm gonna make damn sure everyone else has to suffer with/like me if they want the sweet stuff too! People are bastards.

RandyApril 8, 2011 8:33 AM

Irene Irene is just another spammer pushing their product website.

From what I understand about logical implications, there are two types of fallacy: Affirming the consequence and denying the antecedent. To restate the rain/wet grass scenario: If it rains, the grass will be wet.

I deny the antecedent when I say "It did not rain, ergo the grass is not wet." There are other ways to wet the grass, so the lack of rain says nothing about the state of the grass. I may have turned on the sprinklers.

I affirm the consequence when I say "The grass is wet, ergo it rained." The state of the grass says nothing about whether it rained. Again, the sprinklers may be the reason.

While these are fairly simple examples, in another context the fallacies may not be as evident, and someone skillful in their use may cause you to assume something that wasn't actually said.

...in reference to the 'notP' example illustrating the logic using mortality. in using this particular example you proved the original premise to be incorrect. case in point - jesus. jesus was mortal(because he died) therefore he was a man. jesus is now immortal (i was dead and behold, i am alive forever more) but 'he is still a man'. being imortal does not prove deity. so the example is technically a logical fallacy. ..(a parody - to be taken very seriously)

GreenSquirrelApril 8, 2011 8:55 AM

Why do people think Boston/Plane is arbitrary but vegetables/dessert isnt? (Ignoring suspected spambots)

Rob FunkApril 8, 2011 9:16 AM

@GreenSquirrel,
The vegetables/dessert rule is a common one to grow up with - and to use on people's own children. No it's not a physical rule, but it's an often-familiar one that makes perfect sense to many people (at least in American culture). There's no such feeling about flying vs other modes of transportation to Boston, especially since a huge population is within easy driving (or train) distance. In fact the Boston/plane rule would be entirely counterintuitive to people across the region.

(And IreneIrene is unquestionably a spammer -- just look at the URL on the name.)

ChristopherApril 8, 2011 9:23 AM

I find the attack on the original problem statement to be amusing and instructive. It reminds me of a particular test-taking strategy I learned from hard experience.

I did very well on my Math SAT, not so hot on my Verbal SAT, yet I was considered a good English/ history student. When I stepped back, I realized my problem was that I was *arguing with the questions* instead of providing the answer as it was presented.

In other words, I got in the way of myself. That's my fault. Similarly, people who can't figure out basic rules because they seem arbitrary, are getting in the way of themselves. That this phenomenon is common is very interesting. That it reflects poorly on the original question is a mistake. The goal should not be "make better questions", it should be "make better thinkers". Solving the general case avoids having to solve specific deficiencies down the line.

-C

GreenSquirrelApril 8, 2011 9:38 AM

@Rob Funk

Good points and I agree, but I also think thats why it is a good counter example. Both are really arbitrary but one speaks of a cultural assumption about cheating.

CourtneyApril 8, 2011 9:39 AM

I remember reading that they did a study on dogs and found that dogs have a sense of fairness.

This isn't surprising to anyone who owns two dogs, has them both sit, and gives a treat to only one of them.

Humans are the same way. Fairness is a quality deeply embedded in our brains early on, and given consistent rules, it's usually simple for us to determine fairness.

MusicusApril 8, 2011 10:20 AM

Preacher (On the Bible):

The Bible is the inerrant word of God (Truth). If I believe there can be 1 falsehood, then the book is NOT from God and thus ENTIRELY NOT Truth.

Satanist (On the Anti-Bible):

The Anti-Bible is the inerrant word of Satan (Lies). If I believe there can be 1 Truth, the the book is NOT from Satan and thus ENTIRELY true.

Morale:

Most humans have a hard time dealing with reality. We need to simplify things for our small minds.

MachiineApril 8, 2011 10:25 AM

...and the relevance to security is?

Rob FunkApril 8, 2011 11:05 AM

@GreenSquirrel,
But my point is that there's no (given) reason to assume this is limited to cheating, and it seems reasonable to assume that it's more along the lines of consistency with common assumptions, whether physical or cultural.

@Machiine,
I think the "cheating" part is the relevance to security. Foiling "cheaters" is a goal of security, given a broad definition of "cheaters".

Maybe people do better on tests that are plausible versus scenarios that are absurd. For example, it is absurd to say "Everyone going to Boston" must fly, but plausible to suggest "Only children who eat their vegetables get dessert."
That's the glaring difference between the tests in my opinion. The first test asks the taker to prove the patently absurd, the second test asks the taker to prove the perfectly plausible.

Vege EaterApril 8, 2011 12:44 PM

Very interesting that social rules make such a big difference. But the question has to be phrased very carefully. If you changed: "If a child gets dessert, he or she ate his or her vegetables."
to
"A child gets dessert if he or she ate his vegetables." I bet many people would check the back of the "ate dessert" card to see if the child got it without paying -- but from a propositional logic point of view this is irrelevant.
We naturally invent the punishment side of the rule, i.e. in everyday life these rules are treated as equivalent, but to a logician they are different.

Part of the processing issue may be that the flying to Boston rule is totally arbitrary

Are the results the same if you substitute Hawaii for Boston?

Johnny RojoApril 8, 2011 2:22 PM

Brings new meaning to minding your Ps and Qs.

Shay GuyApril 8, 2011 5:25 PM

Forming the contrapositive in my mind sometimes takes a few seconds, like turning a particularly unwieldy vehicle around. One thing that might be useful to remind people is that is ALSO logically equivalent to ~. Sample statements formulated that way: "Socrates could not have been an immortal man." (Generalization: "There are no immortal men." That requires systems with ∀ and ∃ like first-order logic, though.) "You can't drive to Boston." "You can't drive to Hawaii." "Nobody both refuses to eat their vegetables and gets dessert." "The grass is never dry when it's raining."

It makes a good "midpoint."

KevinApril 8, 2011 6:21 PM

I would just like to point out that there is evidence suggesting that humans detect the oddity and not necessarily the cheater. "Enhanced recognition of defectors depends on their rarity." by Pat Barclay (Cognition vol 107(3) pages 817-828, 2008).

Although this difference is subtle it means that the system is, in fact, more robust and reliable than a true cheater detector would be. A true cheater detector could be overwhelmed by a high cheater to non-cheater ratio but this odd-one-out style of detection makes humans resistant against both a high ratio and changes in the proportion of cheater to non-cheaters, which could otherwise be devastating.

WernerApril 8, 2011 7:39 PM

Test subjects may also be confused about what they're expected to do. E.g., after reading Bruce's description, I wouldn't be quite sure whether to prove/disprove a given P, prove/disprove a given Q, or maybe treat the rule as a mere hypothesis and look for something that tests it.

And even if the problem is presented such that my feeble mind can grasp it, another test subject may still be confused.

The familiarity of the situation also plays into this - if the situation is modeled after a well-known real-life situation, one will not only be familiar with the relation between P and Q, but one will also have an idea of what sort of questions make sense in this context.

Perhaps a better yet still imperfect way to test the hypothesis that it is the cheating that people detect (and not just the general inconsistency) would be to put the problem such that a man-made convention or a fact of nature could define the boundaries. E.g.,

"Alice wants to reach the city. She has to cross the river to get there."
1) "If it is raining, the raging water is too dangerous to cross."
2) "If it is Sunday, the crossing of the holy river is forbidden."

(A less colorful variation of the same theme would be a street and traffic/jaywalking.)

- Werner

AntonApril 8, 2011 9:24 PM

Another lesson here is that if you are going to teach the (P=>Q) => (¬Q=>¬P) rule (which by the way is an axiom not a rule), you will do best if you construct an example to do with cheating.

Actually, Michiel van Lambalgen, a professor at the University of Amsterdam has a different view of the Wason selection task results. His theory is that (in non-cheating situations) people reason using closed world logic, which makes (P -> Q) => (!P -> !Q) perfectly reasonable. People are not "bad at the Wason selection task"; classical logic is just not appropriate for reasoning in real life. (A number of comments above already allude to this.) Consider this line of reasoning:

If Julie has an essay, she studies late in the library
Julie does not have an essay
----
Julie does not study late in the library

Under closed-world logic, this is reasonable: Julie has no business in the library if she does not have an essay.

Michiel van Lambalgen and Keith Stenning published a book on this subject, containing interesting experiment conversation logs. It also addresses the cheating detection theory:
http://www.amazon.com/dp/0262195836
A draft is available online:
http://staff.science.uva.nl/~michiell/docs/...

Russell PollardApril 9, 2011 8:29 AM

I found the examples used when I first studied logic frustrating because they seemed to imply a very small universe of possibilities. In the example in the article this universe seems to contain only two locations, one called Boston and another called New York. If the stated rules for this universe include one location being only accessible by air and the other by road. Then it's fair to assume if somebody is flying in this limited universe they can only be going to one place and if they are driving then they are going to the other.

When a similar example was stated during a lecture I made similar deductions. Then I realised this was too obvious and got suspicious. If it was that easy we wouldn't be told about it in a lecture. So I complicated the situation in the only way I could think of: transpose it to the real universe of possibilities. Most of my classmates were not as cynical and fell into the trap.
In a lecture theatre containing over a hundred students only a hanful got the correct answer. When I asked some of the other students why they felt the answer was something else, they all said they thought the lecturer was only discussing the things that were directly mentioned with no connection to anything in the "real" universe.
Until the person being interviewed realises there are unstated rules in the examples universe, I would expect them to continue to make the same mistake.

If you try to discuss any field with somebody that is not from that field they will assume you are giving them all of the information required to make a reasonable choice. The people being interviewed were not psycologists but they knew it was a psycological test and did not (could not) know what the interviewer was looking for. If you don't state that there are other cities but you have no information on how to get to them, those being interviewed will assume they don't exist.

train = boston || !train = !boston
train = !Boston || !train = Boston
train = !Boston || !train = !Boston
!train ->!Boston

Its a zero sum game, on the face of it you have (1/4) paths(train = Boston is one), but half the paths are inverse, flip to same polarity = (1/2). If one option is correct they both are, and out of two options you have to have taken the train to Boston, or sit and think for ever.
You way up all options and count all options twice expect the one that was given

A information source, half of the people wouldn't trust half would, out of the half that trusted, would the next thing have the same ratio of trust/nontrust???

Gary EApril 10, 2011 1:36 AM

@Russell Pollard

A very good point. It is an obviously arbitrary, made-up world where the only way to travel to Boston is to fly. Why should the reader assume that any other real world features apply?

To later say "The person might have been flying to Boston, New York, San Francisco, or London." is to introduce features that are not part of the original problem.

Bruce SchneierApril 10, 2011 1:26 PM

Typo -- and a particularly embarrassing one at that -- fixed.

Bruce SchneierApril 10, 2011 1:28 PM

"...and the relevance to security is?"

I've been doing a lot of reading about our natural physiological security systems. This is a natural system to detect cheaters.

@Bruce Schneier, are you writing a book, or could you post some ideas?

Is it possible for the reason for the difference to be that people automatically associate a rationale for the desert story, but it is not so easy for the plane-to-Boston story?

I suspect that if you present the problem with some rationale, like "there is information about a possible plane bombing targeting Boston", then people may perform better than the original logic problem.

If that is the case, then the security blindspot may be smaller than the experiment suggests, since security problems have concrete contexts and an understanding of what we try to protect and why.

Shay GuyApril 10, 2011 11:13 PM

...OK, let's try that again with parentheses instead of angle brackets.

One thing that might be useful to remind people is that (P→Q) is ALSO logically equivalent to ~(P^~Q).Sample statements formulated that way: "Socrates could not have been an immortal man." (Generalization: "There are no immortal men." That requires systems with ∀ and ∃ like first-order logic, though.) "You can't drive to Boston." "You can't drive to Hawaii." "Nobody both refuses to eat their vegetables and gets dessert." "The grass is never dry when it's raining."

@Julien Couvreur , probably but telling people that the correct solution is more in one direction won't revels weather they are cheaters or not. If you don't know all the possible information ,in theory the only right answers should be correct without thinking of the answer given(use theirs), or I can't work it out.
Anything other answer would not have got to a full ending, so they are throwing out the farthest chess moves ahead that there wiring can achieve and risking there answer to get some advantage over the interviewer.

By chance a some cheaters should be hiding by lucky guessing a answer that the interview expects to hear

Clive RobinsonApril 11, 2011 12:13 AM

@ Bruce,

"Typo -- and a particularly embarrassing one at that -- fixed"

Your being unusally coy on this (ie no "Edited..."). So I'm assuming it was not just an S making the difference between sand and sweet...

I think the comments show in general how many smart people are bad at propositional logic. (I'm assuming most of the comments are from smart people: apart from the spammer.)

A lot of it seems to be a failure to map an algebraic expression like P → Q into an *exactly equivalent* verbal description, followed by similar loss of precision in reasoning each step verbally.

That's why mathematical proofs and engineering calculations aren't in the form of prose essays.

GreenSquirrelApril 11, 2011 9:11 AM

I dont want to beat a dead horse but...

Yes, the flying to Boston rule it totally arbitrary. It really is.

However, so is the "must eat vegetables to get dessert." It really is, even if it speaks of something we have been subjected to as children. It is an arbitrary rule.

When we look at the flying to Boston, it seem strange. There is no implication of someone having cheated to get round it. It seems obviously arbitrary because there is no apparent loss or benefit from abiding by it or circumventing it. As a result, we scratch our heads and say "but why cant he drive?"

Now, when it comes to the vegetable and dessert, we appear to think differently. We dont scratch our heads and say "what if it was a barbeque and there were no vegetables" or "what if the parents relented today." We dont even question the odd standard which seems to assume children dont like vegetables. We see an instant situation where someone *may* have circumvented a culturally apt rule for personal gain.

We can use post-hoc rationalisations all day, but the fact is both are arbitrary. Both are simply things someone has asserted as being a rule.

The implication here is that, for whatever reason and by whatever mechanism, we are indoctrinated to be more astute when a rule breaker has done something we would consider cheating.

Andre P.April 11, 2011 2:07 PM

@Russell Pollard/Gary E
"they thought the lecturer was only discussing the things that were directly mentioned with no connection to anything in the "real" universe."

The point to logic puzzles is that it doesn't matter that you assume it is "real world" or not. Your lecturer did you a disservice, or you misunderstood the given assumptions, because pure logic puzzles don't have to make sense. When someone asks you to draw conclusions about certain statements, and doesn't allow you to ask questions, then you should always base your answer on the given facts.

I (think I) once got dismissed as a potential juror (by the prosecution) because I was the only one who thought that being guilty of one crime didn't necessarily mean that the defendant was more likely to be guilty of a second crime. Given the question, as stated, I could not draw a conclusion about that particular defendant's likelihood of being guilty of the secondary crime; unless the prosecution could prove a correlation in this case, it would be a disservice to the defendant to assume a correlation based upon other convicted defendants, especially without knowing how many others were unjustly convicted. The problem is that it's the same amount of work to prove guilt in the second crime as it is to prove correlation between the first and second crimes in a single case.

Some of you may disagree with my not having put on my "real world" hat, but imagine if the defendant was you.

spammerApril 11, 2011 4:35 PM

If someone has tried and flowing to Hawaii, they are "aware" that's the only way to get there(high chance they could cheat). If someone hasn't tried or gone there, then they "trust" what someone has told them(less likely to cheat)
risk/reward
!p->q ,p=!p->!q

Does this work if you the conditional is the opposite of what you expect from cheating? Like:
If a child gets dessert, he or she did not eat his or her vegetables

In this case, the social cost is not paid.

AngusApril 11, 2011 7:54 PM

-- If you know "if P, then Q" and "not Q," then you know "not P."

This is easy to follow if P=COCKER SPANIEL and Q=DOG. Even SOCRATES/MORTAL is too abstract.

If the general rule was "If an animal is a COCKER SPANIEL then it is a DOG." and the cards were labelled DOG, CAT, COCKER SPANIEL, SIAMESE it would be a lot easier to pick which cards you have to flip to check: I don't care about cats. Only dogs.

Before I pronounce on a question of formal logic I always give it a quick Cocker Spaniel test.

Putting it in terms of fairness is just like putting it in terms of Cocker Spaniels: it's no longer arbitrary - and that's what makes it easier to solve.

@Mike, the child might get sick eating vegetables. If you were the interviewer would you say the social cost isn't being paid, if they didn't tell you that.
What would you say if anything could happen to get "If a child gets dessert, he or she did not eat his or her vegetables" to rule out cheating

Oh dear, Wason again :-)

Responses to two main things somebody somewhere will eventually claim:

1. Wason's selection test is not easier with "meaningful" content, whatever that means. There are versions with "real world" content - actually you use one - which people still fail at.

2. The Cosmides version is not easier because of social exchange. (It helps if the conditional is deontic - see below.)

3. It is also not satisfactorily explained by appeal to an "innate" cheater detection module. (Why do most people have two nostrils? Easy! It's innate!)

See this paper for one nice explanation of what's going on:

Key point:

"In the ordinary descriptive versions, the truth or falsity of the rule is what is in question, and what the cards selected are supposed to help establish. In the deontic versions, on the other hand, the truth of the rule (i.e., the fact that the rule is in force) is treated as axiomatic, and what subjects are expected to look for is not evidence of truth or falsity, but evidence of violation."

(They give a recipe for making many other versions which are easier.)

Likely that the version:

"If a child gets dessert, he or she ate his or her vegetables."

won't work in cultures where the rule is not obviously true.

Also I disagree that the (classical logic interpreted descriptive) task tests modus ponens and modus tollens.

Take MP; you need P->Q and P and conclude Q.

You are given an incomplete truthtable; for some rows (i.e., cards) it's possible to tell the truth value of the conditional without more information (when the antecdent is false or the consequent is true).

Of course this can be mapped into a task about MP/MT, etc, but I don't think that explains what's going on.

Also: beware undergraduate psychology textbook introductions to the task, they seem to be decades behind the psych of reasoning literature!

A further complication, while I'm at it. According to Oaksford and Chater, and others, it's possible to justify the "error" people make on the standard descriptive task by appeal to the Rev Bayes.

Derick van HeerdenApril 15, 2011 4:39 AM

Not sure that the statement of the rule, If a child gets dessert, he or she ate his or her vegetables, is best put this way. It looks more like an incomplete argument with an unstated premise (and the unstated premise actually expresses the rule or conditional statement). This becomes apparent by replacing the "if" with a "since" or "because" and inserting a "it follows" after the comma. The sentence then reads as follows: "since [the] child gets dessert, it follows that he or she ate his or her vegetables." So, you have a conclusion: Q, based on evidence P. making P -> Q, the unstated premise. The rule, therefore, is 'If a child eats dessert, then the child has eaten her veg' which is not a great rule. The rule should really be 'If a child eats her veg, then she gets dessert.' But if that is the rule, then the argument is technically invalid, having the form: P -> Q, Q, Therefore P.

I don't understand this "Modus tollens" stuff. Perhaps someone can explain where I'm going wrong in this Python session...

>>> def test(P=None, Q=None):
... print 'when P is %s and Q is %s' % (P, Q),
... # "If P, then Q"
... if P is True: Q = True
... print '\tP: %s, Q: %s' % (P, Q)
...
>>> test(P=True) # P. Therefore, Q.
when P is True and Q is None P: True, Q: True
>>> test(Q=False) # Not Q. Therefore, not P.
when P is None and Q is False P: None, Q: False

Toby Ovod-EverettApril 15, 2011 8:45 AM

If you eat your vegetables, then you get dessert.

That is a loose expression of "dessert -> vegetables", which is, as Derick van Heerden pointed out, backwards. What's weird is that "If you commit the crime, then you go to jail" is crime -> jail. It seems like the simple if then construction in English depends upon whether the "then" is a may or a must (and desireable vs. non-desireable).

But this one is easy:
If you are pregnant, then you had sex.
That is, "pregnant -> sex".

The more I think about this, the more I realize that English is not always well suited to expressing "implies" (well, I sort of already knew this).

Here's another one for you . . .
If you have brown eyes, one of your (biological) parents must have had brown eyes.

That one almost tripped up my mother while teaching genetics to some Junior High school kids back in the 60s. "But I have brown eyes and both my parents have blue eyes?" "Well, ummmmm, . . ." After class the student did mention, "But I was adopted".

Interesting observation! I agree that the reason why most people were not able to solve the first test about the flight to Boston is the fact that they couldn’t find anything important in this case.

However, everyone cares about justice. This is why most people had no difficulties with the second test.

Human beings are always trying to detect something illegal, false, unfair, and so on, because they know that most people lie and disobey rules. This was a very simple task.

Clive RobinsonFebruary 8, 2012 5:54 AM

@ Bruce, MODERATOR,

On seeing the obviously spam / link advert post above (#c696651) of February 8, 2012 4:23 AM, it is also obvious that the originator has made a mistake and included a whole load (if not all) of their "stock phrases" used for "cutting-n-pasting" into posts.

Now the thought occurs that these phrases can be used not just to filter out the offending link adverts but also for research.

On the assumption that the phrase list is effectivly unique to a link spam organisation a search on Google could identify the client list pluss the blog sites being targeted.

Thus on the principle of "know thy enemy" it may be possible to "trace the money" to identify the spamers supporting infrastructure, thus "knowing thy enemies friends".

If other similar research is to be belived most of these spaming organisations use one or two of a very small handfull of support organisations.

So at the end of the day whilst it may not be possible to get at the actual spammers it may be possible to get at their support organisations in some way even if it is simply to "name and shame" and make their status sufficiently pariah like that it starts to effect their bottom line.

E-mail is optional and will not be displayed on the site.

Remember Me?

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>