Punishment and Trust

Interesting research: “Third-party punishment as a costly signal of trustworthiness, by Jillian J. Jordan, Moshe Hoffman, Paul Bloom,and David G. Rand, Nature:

Abstract: Third-party punishment (TPP), in which unaffected observers punish selfishness, promotes cooperation by deterring defection. But why should individuals choose to bear the costs of punishing? We present a game theoretic model of TPP as a costly signal of trustworthiness. Our model is based on individual differences in the costs and/or benefits of being trustworthy. We argue that individuals for whom trustworthiness is payoff-maximizing will find TPP to be less net costly (for example, because mechanisms that incentivize some individuals to be trustworthy also create benefits for deterring selfishness via TPP). We show that because of this relationship, it can be advantageous for individuals to punish selfishness in order to signal that they are not selfish themselves. We then empirically validate our model using economic game experiments. We show that TPP is indeed a signal of trustworthiness: third-party punishers are trusted more, and actually behave in a more trustworthy way, than non-punishers. Furthermore, as predicted by our model, introducing a more informative signal—the opportunity to help directly—attenuates these signalling effects. When potential punishers have the chance to help, they are less likely to punish, and punishment is perceived as, and actually is, a weaker signal of trustworthiness. Costly helping, in contrast, is a strong and highly used signal even when TPP is also possible. Together, our model and experiments provide a formal reputational account of TPP, and demonstrate how the costs of punishing may be recouped by the long-run benefits of signalling one’s trustworthiness.

More accessible essay.

Tags: academic papers, psychology of security, trust

Posted on March 14, 2016 at 12:59 PM • 12 Comments

Comments

Worried iPhone User • March 14, 2016 1:38 PM

Mr. Schneier,

I apologize for posting off-topic, however I was very much hoping to ask you about iPhone encryption. I recently decided to see if plugging my locked iPhone into my computer via USB (running the Ubuntu OS) would enable me to access any data. I thought with it being currently locked it would deny any connection, or at least not allow me to see any data. To my surprise I saw numerous folders pop up unencrypted, including my Photos folder which I could access, again without inputting my passcode. I could do this even though Apple has stated that even your pictures are now encrypted with iOS 8 or later. I have the latest version of iOS installed. I found this website which duplicated this issue, but this post is several years old. It appears that Apple is all hype and have not actually secured their phones as far as I can see. Or am I missing something?

Jesse Thompson • March 14, 2016 2:32 PM

Hmm. In a different arena of study — Dating Theory — “the opportunity to help directly as a strong signal of trustworthiness” seems instead to be an easily exploited mechanism leading to “The Friendzone”. Or converted back to more general trust analysis terms, this signal can be answered by exploitative relationships promising (or at least hinting at and then never explicitly denying) greater mutually-profitable cooperation in the future that then never materializes.

Hell, the company I work for has even encountered this effect a number of times with other communications companies in this area. It makes me think that above “offering help” or being helpful, there must be some well evolved strategy to actually offer that help maximally while also minimizing risk to self of damage or of afore-mentioned extortion.

Someone • March 14, 2016 6:43 PM

@Jesse,

Within dating, the people who are friendzoned often send an overly strong signal of trust and a commitment to cooperation even in the face of defection. It is hard to not exploit the party in such situations. One might try to make a small defection in hopes of the other party recalibrating trust levels to something more appropriate, but if the other party continues to ignore the defection and always cooperate, what can you do?

Clive Robinson • March 14, 2016 6:45 PM

The majority of the authors have written another paper that at a quick view appears to be a similar take on the same subjects.

However this second paper is rather easier to download,

http://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID2725550_code2225468.pdf?abstractid=2725550&mirid=1

Just passin' thru • March 14, 2016 11:47 PM

… brings new meaning to the acronym TPP. (I’ve been conditioned to understand it as Trans-Pacific Partnership). Talk about Third Party Punishment!

paul • March 15, 2016 8:30 AM

Seems to me that TPP (ahem) as a signal of trustworthiness could also go under the heading of clearing out the competition. A lot depends on how good your information is about whether someone has defected and how many iterations there are in the game.

A. Sinan Unur • March 15, 2016 9:47 AM

I am used to people re-discovering the Mafia every so often. People are led to think about Prisoners’ Dilemma in terms of altruism and social good, but they forget the original motivation was about getting two criminals to confess. As a society, we don’t want criminals to co-operate and beat Prisoners’ Dilemma: We want the exact opposite of it.

Yet, that is the context in which most of these enforcement mechanisms that steer players away from the dominant strategy equilibrium to the Pareto optimum for them. Once again, PD has made so many social scientists career that they forget about the context of the game, and there other individuals whose utility depends on the outcome of the game.

Anyway, back to the Mafia, or, in general, organized crime. They are effective because they can set up these third party punishment networks. If you confess, you expect grave harm to come to yourself or your loved ones. Therefore, co-operation with each other (i.e. co-operation against the rest of the society) becomes second nature for members of a criminal network. Every time these prisoners play PD, the move away from the unique equilibrium of the game to the optimum.

This only breaks down when someone on the side of the “good guys” starts pursuing the top cadres of the organization effectively, thereby leading to the breakdown of the assurance that those who move to the equilibrium strategy will be punished.

In international relationships, where people also love to resort to Prisoners’ Dilemma to explain problems as one of “trust”, the real problem is the non-existence of a third-party punishment. Getting some decision in the UN security council or international courts does not remove the responsibility of one of the players in the game to undertake the punishment of the other party.

And, punishment is costly on the party doing the punishment. Therein lies the rub: You cannot have “diplomacy” without your adversaries unwavering belief in your ability and willingness to obliterate them.

If you take this a little further, you realize that, in most instances, the correct game to model a conflict is the Game of Chicken, and not PD. GoC has three equilibria, one of which is probabilistic, and therefore the outcome depends on the beliefs of the players.

This is too hard for most people, and I mean those with PhDs also, to grasp. PD is comfortable: Just one equilibrium, and one optimum. They are different. OMG! Therefore we must have men with sticks patrolling the subways so no one spits on the platform.

paul • March 15, 2016 10:39 AM

@A. Sinan Unur

I think it may be a cultural or a generational thing. If you learned PD in the 70s or 80s in the US, there was an implicit subtext (even if not stated or explicitly contradicted in the hypothetical as given) that the thing the prisoners had been arrested for might or might not be what most people consider a serious crime (or even a crime at all) and the the police didn’t care so much about finding a guilty party as finding someone to put in jail. So third-party punishment by a supervising criminal organization wouldn’t have been high on the list of contextual associations, even though it does make a lot of sense.

I also wonder whether a mafia can properly be considered an independent TPP, insofar as (in its stereoytpical manifestation) everyone in the game is either a member or pays some kind of tribute to it. But it definitely played that kind of third-party role. (When I was in college, there was one street in the non-ritzy neighborhood where you could part a car with no worry of having it broken into; and the common belief was the Somebody’s Grandmother lived on that street.)

David • March 15, 2016 10:23 PM

This “Study” wins this week’s contest for stating the obvious in the most number of words.

name.withheld.for.obvious.reasons • March 16, 2016 11:49 PM

Two quick thoughts…one question and a statement.

1.) Isn’t China experimenting with this to create a cultural of compliance?

2.) In Nazi Germany, state identity (what it means to be a “Good German”) was giving the ultimate “Label of Trust”. The 3rd Reich (notice this is not a Trump’ism, aka the Reich 3) had managed to institutionalize trust in both explicit and implicit (as is the case in this “Trust” model) forms of authoritative trust. Here is where I continue to argue about the role of honesty. Research like this is “weighted” to generate a empirical model to support one or another “hypothesis” as most of this type of research fails to reach theoretical rigor.

Worried iPhone User • March 21, 2016 11:22 PM

Mr. Schneier,

I believe I might have figured out my own problem to the question I asked above. I had forgotten that I had set my iPhone to ‘Trust’ my computer, which I read while trying to learn more will allow the trusted computer to access the data unencrypted. This would not happen on any other computer that I had not authorized, according to the information I read. I am still new to encryption and the whole bit. But I’m learning. 🙂 PS. Read Data and Goliath and it was FANTASTIC. Thank you.

Jesse Thompson • May 30, 2018 2:26 PM

@Someone

Well, since they already trust you you can simply flat out tell them about the problem.

Unfortunately, in social situations usually 0.0 of the people involved are capable of sufficient self-reflection to even attach words to the intuitions which you describe. So even if they wanted to tell the doormat how to not doormat, they wouldn’t know what to say.

Punishment and Trust

Comments

Leave a comment Cancel reply