Entries Tagged "trust"

Page 2 of 14

Punishment and Trust

Interesting research: “Third-party punishment as a costly signal of trustworthiness, by Jillian J. Jordan, Moshe Hoffman, Paul Bloom,and David G. Rand, Nature:

Abstract: Third-party punishment (TPP), in which unaffected observers punish selfishness, promotes cooperation by deterring defection. But why should individuals choose to bear the costs of punishing? We present a game theoretic model of TPP as a costly signal of trustworthiness. Our model is based on individual differences in the costs and/or benefits of being trustworthy. We argue that individuals for whom trustworthiness is payoff-maximizing will find TPP to be less net costly (for example, because mechanisms that incentivize some individuals to be trustworthy also create benefits for deterring selfishness via TPP). We show that because of this relationship, it can be advantageous for individuals to punish selfishness in order to signal that they are not selfish themselves. We then empirically validate our model using economic game experiments. We show that TPP is indeed a signal of trustworthiness: third-party punishers are trusted more, and actually behave in a more trustworthy way, than non-punishers. Furthermore, as predicted by our model, introducing a more informative signal — the opportunity to help directly — attenuates these signalling effects. When potential punishers have the chance to help, they are less likely to punish, and punishment is perceived as, and actually is, a weaker signal of trustworthiness. Costly helping, in contrast, is a strong and highly used signal even when TPP is also possible. Together, our model and experiments provide a formal reputational account of TPP, and demonstrate how the costs of punishing may be recouped by the long-run benefits of signalling one’s trustworthiness.

More accessible essay.

Posted on March 14, 2016 at 12:59 PMView Comments

Lots More Writing about the FBI vs. Apple

I have written two posts on the case, and at the bottom of those essays are lots of links to other essays written by other people. Here are more links.

If you read just one thing on the technical aspects of this case, read Susan Landau’s testimony before the House Judiciary Committee. It’s very comprehensive, and very good.

Others are testifying, too.

Apple is fixing the vulnerability. The Justice Department wants Apple to unlock nine more phones.

Apple prevailed in a different iPhone unlocking case.

Why the First Amendment is a bad argument. And why the All Writs Act is the wrong tool.

Dueling poll results: Pew Research reports that 51% side with the FBI, while a Reuters poll reveals that “forty-six percent of respondents said they agreed with Apple’s position, 35 percent said they disagreed and 20 percent said they did not know,” and that “a majority of Americans do not want the government to have access to their phone and Internet communications, even if it is done in the name of stopping terror attacks.”

One of the worst possible outcomes from this story is that people stop installing security updates because they don’t trust them. After all, a security update mechanism is also a mechanism by which the government can install a backdoor. Here’s one essay that talks about that. Here’s another.

Cory Doctorow comments on the FBI’s math denialism. Yochai Benkler sees this as a symptom of a greater breakdown in government trust. More good commentary from Jeff Schiller, Julian Sanchez, and Jonathan Zdziarski. Marcy Wheeler’s comments. Two posts by Dan Wallach. Michael Chertoff and associates weigh in on the side of security over surveillance.

Here’s a Catholic op-ed on Apple’s side. Bill Gates sides with the FBI. And a great editorial cartoon.

Here’s high snark from Stewart Baker. Baker asks some very good (and very snarky) questions. But the questions are beside the point. This case isn’t about Apple or whether Apple is being hypocritical, any more than climate change is about Al Gore’s character. This case is about the externalities of what the government is asking for.

One last thing to read.

Okay, one more, on the more general back door issue.

EDITED TO ADD (3/2): Wall Street Journal editorial. And here’s video from the House Judiciary Committee hearing. Skip to around 34:50 to get to the actual beginning.

EDITED TO ADD (3/3): Interview with Rep. Darrell Issa. And at the RSA Conference this week, both Defense Secretary Ash Carter and Microsoft’s chief legal officer Brad Smith sided with Apple against the FBI.

EDITED TO ADD (3/4): Comments on the case from the UN High Commissioner for Human Rights.

EDITED TO ADD (3/7): Op ed by Apple. And an interesting article on the divide in the Obama Administration.

EDITED TO ADD (3/10): Another good essay.

EDITED TO ADD (3/13): President Obama’s comments on encryption: he wants back doors. Cory Doctorow reports.

Posted on March 1, 2016 at 6:47 AMView Comments

NSA Reorganizing

The NSA is undergoing a major reorganization, combining its attack and defense sides into a single organization:

In place of the Signals Intelligence and Information Assurance directorates ­ the organizations that historically have spied on foreign targets and defended classified networks against spying, respectively ­ the NSA is creating a Directorate of Operations that combines the operational elements of each.

It’s going to be difficult, since their missions and culture are so different.

The Information Assurance Directorate (IAD) seeks to build relationships with private-sector companies and help find vulnerabilities in software ­ most of which officials say wind up being disclosed. It issues software guidance and tests the security of systems to help strengthen their defenses.

But the other side of the NSA house, which looks for vulnerabilities that can be exploited to hack a foreign network, is much more secretive.

“You have this kind of clash between the closed environment of the sigint mission and the need of the information-assurance team to be out there in the public and be seen as part of the solution,” said a second former official. “I think that’s going to be a hard trick to pull off.”

I think this will make it even harder to trust the NSA. In my book Data and Goliath, I recommended separating the attack and defense missions of the NSA even further, breaking up the agency. (I also wrote about that idea here.)

And missing in their reorg is how US CyberCommmand’s offensive and defensive capabilities relate to the NSA’s. That seems pretty important, too.

EDITED TO ADD (2/11): Some more commentary.

EDITED TO ADD (2/13): Another.

Posted on February 5, 2016 at 3:15 PMView Comments

How the CIA Might Target Apple's XCode

The Intercept recently posted a story on the CIA’s attempts to hack the iOS operating system. Most interesting was the speculation that it hacked XCode, which would mean that any apps developed using that tool would be compromised.

The security researchers also claimed they had created a modified version of Apple’s proprietary software development tool, Xcode, which could sneak surveillance backdoors into any apps or programs created using the tool. Xcode, which is distributed by Apple to hundreds of thousands of developers, is used to create apps that are sold through Apple’s App Store.

The modified version of Xcode, the researchers claimed, could enable spies to steal passwords and grab messages on infected devices. Researchers also claimed the modified Xcode could “force all iOS applications to send embedded data to a listening post.” It remains unclear how intelligence agencies would get developers to use the poisoned version of Xcode.

Researchers also claimed they had successfully modified the OS X updater, a program used to deliver updates to laptop and desktop computers, to install a “keylogger.”

It’s a classic application of Ken Thompson’s classic 1984 paper, “Reflections on Trusting Trust,” and a very nasty attack. Dan Wallach speculates on how this might work.

Posted on March 16, 2015 at 7:38 AMView Comments

Over 700 Million People Taking Steps to Avoid NSA Surveillance

There’s a new international survey on Internet security and trust, of “23,376 Internet users in 24 countries,” including “Australia, Brazil, Canada, China, Egypt, France, Germany, Great Britain, Hong Kong, India, Indonesia, Italy, Japan, Kenya, Mexico, Nigeria, Pakistan, Poland, South Africa, South Korea, Sweden, Tunisia, Turkey and the United States.” Amongst the findings, 60% of Internet users have heard of Edward Snowden, and 39% of those “have taken steps to protect their online privacy and security as a result of his revelations.”

The press is mostly spinning this as evidence that Snowden has not had an effect: “merely 39%,” “only 39%,” and so on. (Note that these articles are completely misunderstanding the data. It’s not 39% of people who are taking steps to protect their privacy post-Snowden, it’s 39% of the 60% of Internet users — which is not everybody — who have heard of him. So it’s much less than 39%.)

Even so, I disagree with the “Edward Snowden Revelations Not Having Much Impact on Internet Users” headline. He’s having an enormous impact. I ran the actual numbers country by country, combining data on Internet penetration with data from this survey. Multiplying everything out, I calculate that 706 million people have changed their behavior on the Internet because of what the NSA and GCHQ are doing. (For example, 17% of Indonesians use the Internet, 64% of them have heard of Snowden and 62% of them have taken steps to protect their privacy, which equals 17 million people out of its total 250-million population.)

Note that the countries in this survey only cover 4.7 billion out of a total 7 billion world population. Taking the conservative estimates that 20% of the remaining population uses the Internet, 40% of them have heard of Snowden, and 25% of those have done something about it, that’s an additional 46 million people around the world.

It’s probably true that most of those people took steps that didn’t make any appreciable difference against an NSA level of surveillance, and probably not even against the even more pervasive corporate variety of surveillance. It’s probably even true that some of those people didn’t take steps at all, and just wish they did or wish they knew what to do. But it is absolutely extraordinary that 750 million people are disturbed enough about their online privacy that they will represent to a survey taker that they did something about it.

Name another news story that has caused over ten percent of the world’s population to change their behavior in the past year? Cory Doctorow is right: we have reached “peak indifference to surveillance.” From now on, this issue is going to matter more and more, and policymakers around the world need to start paying attention.

Related: a recent Pew Research Internet Project survey on Americans’ perceptions of privacy, commented on by Ben Wittes.

This essay previously appeared on Lawfare.

EDITED TO ADD (12/15): Reddit thread.

EDITED TO ADD (12/16): Slashdot thread.

EDITED TO ADD (1/23): This essay has been translated into German.

Posted on December 15, 2014 at 6:07 AMView Comments

Internet Subversion

In addition to turning the Internet into a worldwide surveillance platform, the NSA has surreptitiously weakened the products, protocols, and standards we all use to protect ourselves. By doing so, it has destroyed the trust that underlies the Internet. We need that trust back.

Trust is inherently social. It is personal, relative, situational, and fluid. It is not uniquely human, but it is the underpinning of everything we have accomplished as a species. We trust other people, but we also trust organizations and processes. The psychology is complex, but when we trust a technology, we basically believe that it will work as intended.

This is how we technologists trusted the security of the Internet. We didn’t have any illusions that the Internet was secure, or that governments, criminals, hackers, and others couldn’t break into systems and networks if they were sufficiently skilled and motivated. We didn’t trust that the programmers were perfect, that the code was bug-free, or even that our crypto math was unbreakable. We knew that Internet security was an arms race, and the attackers had most of the advantages.

What we trusted was that the technologies would stand or fall on their own merits.

We now know that trust was misplaced. Through cooperation, bribery, threats, and compulsion, the NSA — and the United Kingdom’s GCHQ — forced companies to weaken the security of their products and services, then lie about it to their customers.

We know of a few examples of this weakening. The NSA convinced Microsoft to make some unknown changes to Skype in order to make eavesdropping on conversations easier. The NSA also inserted a degraded random number generator into a common standard, then worked to get that generator used more widely.

I have heard engineers working for the NSA, FBI, and other government agencies delicately talk around the topic of inserting a “backdoor” into security products to allow for government access. One of them told me, “It’s like going on a date. Sex is never explicitly mentioned, but you know it’s on the table.” The NSA’s SIGINT Enabling Project has a $250 million annual budget; presumably it has more to show for itself than the fragments that have become public. Reed Hundt calls for the government to support a secure Internet, but given its history of installing backdoors, why would we trust claims that it has turned the page?

We also have to assume that other countries have been doing the same things. We have long believed that networking products from the Chinese company Huawei have been backdoored by the Chinese government. Do we trust hardware and software from Russia? France? Israel? Anywhere?

This mistrust is poison. Because we don’t know, we can’t trust any of them. Internet governance was largely left to the benign dictatorship of the United States because everyone more or less believed that we were working for the security of the Internet instead of against it. But now that system is in turmoil. Foreign companies are fleeing US suppliers because they don’t trust American firms’ security claims. Far worse governments are using these revelations to push for a more isolationist Internet, giving them more control over what their citizens see and say.

All so we could eavesdrop better.

There is a term in the NSA: “nobus,” short for “nobody but us.” The NSA believes it can subvert security in such a way that only it can take advantage of that subversion. But that is hubris. There is no way to determine if or when someone else will discover a vulnerability. These subverted systems become part of our infrastructure; the harms to everyone, once the flaws are discovered, far outweigh the benefits to the NSA while they are secret.

We can’t both weaken the enemy’s networks and protect our own. Because we all use the same products, technologies, protocols, and standards, we either allow everyone to spy on everyone, or prevent anyone from spying on anyone. By weakening security, we are weakening it against all attackers. By inserting vulnerabilities, we are making everyone vulnerable. The same vulnerabilities used by intelligence agencies to spy on each other are used by criminals to steal your passwords. It is surveillance versus security, and we all rise and fall together.

Security needs to win. The Internet is too important to the world — and trust is too important to the Internet — to squander it like this. We’ll never get every power in the world to agree not to subvert the parts of the Internet they control, but we can stop subverting the parts we control. Most of the high-tech companies that make the Internet work are US companies, so our influence is disproportionate. And once we stop subverting, we can credibly devote our resources to detecting and preventing subversion by others.

This essay previously appeared in the Boston Review.

Posted on May 12, 2014 at 6:26 AMView Comments

Is Google Too Big to Trust?

Interesting essay about how Google’s lack of transparency is hurting their trust:

The reality is that Google’s business is and has always been about mining as much data as possible to be able to present information to users. After all, it can’t display what it doesn’t know. Google Search has always been an ad-supported service, so it needs a way to sell those users to advertisers — that’s how the industry works. Its Google Now voice-based service is simply a form of Google Search, so it too serves advertisers’ needs.

In the digital world, advertisers want to know more than the 100,000 people who might be interested in buying a new car. They now want to know who those people are, so they can reach out to them with custom messages that are more likely to be effective. They may not know you personally, but they know your digital persona — basically, you. Google needs to know about you to satisfy its advertisers’ demands.

Once you understand that, you understand why Google does what it does. That’s simply its business. Nothing is free, so if you won’t pay cash, you’ll have to pay with personal information. That business model has been around for decades; Google didn’t invent that business model, but Google did figure out how to make it work globally, pervasively, appealingly, and nearly instantaneously.

I don’t blame Google for doing that, but I blame it for being nontransparent. Putting unmarked sponsored ads in the “regular” search results section is misleading, because people have been trained by Google to see that section of the search results as neutral. They are in fact not. Once you know that, you never quite trust Google search results again. (Yes, Bing’s results are similarly tainted. But Microsoft never promised to do no evil, and most people use Google.)

Posted on April 24, 2014 at 6:45 AMView Comments

Sidebar photo of Bruce Schneier by Joe MacInnis.