Entries Tagged "surveillance"

Page 83 of 93

Terrorists, Data Mining, and the Base Rate Fallacy

I have already explained why NSA-style wholesale surveillance data-mining systems are useless for finding terrorists. Here’s a more formal explanation:

Floyd Rudmin, a professor at a Norwegian university, applies the mathematics of conditional probability, known as Bayes’ Theorem, to demonstrate that the NSA’s surveillance cannot successfully detect terrorists unless both the percentage of terrorists in the population and the accuracy rate of their identification are far higher than they are. He correctly concludes that “NSA’s surveillance system is useless for finding terrorists.”

The surveillance is, however, useful for monitoring political opposition and stymieing the activities of those who do not believe the government’s propaganda.

And here’s the analysis:

What is the probability that people are terrorists given that NSA’s mass surveillance identifies them as terrorists? If the probability is zero (p=0.00), then they certainly are not terrorists, and NSA was wasting resources and damaging the lives of innocent citizens. If the probability is one (p=1.00), then they definitely are terrorists, and NSA has saved the day. If the probability is fifty-fifty (p=0.50), that is the same as guessing the flip of a coin. The conditional probability that people are terrorists given that the NSA surveillance system says they are, that had better be very near to one (p=1.00) and very far from zero (p=0.00).

The mathematics of conditional probability were figured out by the Scottish logician Thomas Bayes. If you Google “Bayes’ Theorem”, you will get more than a million hits. Bayes’ Theorem is taught in all elementary statistics classes. Everyone at NSA certainly knows Bayes’ Theorem.

To know if mass surveillance will work, Bayes’ theorem requires three estimations:

  1. The base-rate for terrorists, i.e. what proportion of the population are terrorists;
  2. The accuracy rate, i.e., the probability that real terrorists will be identified by NSA;
  3. The misidentification rate, i.e., the probability that innocent citizens will be misidentified by NSA as terrorists.

No matter how sophisticated and super-duper are NSA’s methods for identifying terrorists, no matter how big and fast are NSA’s computers, NSA’s accuracy rate will never be 100% and their misidentification rate will never be 0%. That fact, plus the extremely low base-rate for terrorists, means it is logically impossible for mass surveillance to be an effective way to find terrorists.

I will not put Bayes’ computational formula here. It is available in all elementary statistics books and is on the web should any readers be interested. But I will compute some conditional probabilities that people are terrorists given that NSA’s system of mass surveillance identifies them to be terrorists.

The US Census shows that there are about 300 million people living in the USA.

Suppose that there are 1,000 terrorists there as well, which is probably a high estimate. The base-rate would be 1 terrorist per 300,000 people. In percentages, that is .00033%, which is way less than 1%. Suppose that NSA surveillance has an accuracy rate of .40, which means that 40% of real terrorists in the USA will be identified by NSA’s monitoring of everyone’s email and phone calls. This is probably a high estimate, considering that terrorists are doing their best to avoid detection. There is no evidence thus far that NSA has been so successful at finding terrorists. And suppose NSA’s misidentification rate is .0001, which means that .01% of innocent people will be misidentified as terrorists, at least until they are investigated, detained and interrogated. Note that .01% of the US population is 30,000 people. With these suppositions, then the probability that people are terrorists given that NSA’s system of surveillance identifies them as terrorists is only p=0.0132, which is near zero, very far from one. Ergo, NSA’s surveillance system is useless for finding terrorists.

Suppose that NSA’s system is more accurate than .40, let’s say, .70, which means that 70% of terrorists in the USA will be found by mass monitoring of phone calls and email messages. Then, by Bayes’ Theorem, the probability that a person is a terrorist if targeted by NSA is still only p=0.0228, which is near zero, far from one, and useless.

Suppose that NSA’s system is really, really, really good, really, really good, with an accuracy rate of .90, and a misidentification rate of .00001, which means that only 3,000 innocent people are misidentified as terrorists. With these suppositions, then the probability that people are terrorists given that NSA’s system of surveillance identifies them as terrorists is only p=0.2308, which is far from one and well below flipping a coin. NSA’s domestic monitoring of everyone’s email and phone calls is useless for finding terrorists.

As an exercise to the reader, you can use the same analysis to show that data mining is an excellent tool for finding stolen credit cards, or stolen cell phones. Data mining is by no means useless; it’s just useless for this particular application.

Posted on July 10, 2006 at 7:15 AM • View Comments

Annual Report from the Privacy Commissioner of Canada

Excellent reading.

It is my duty, in this Annual Report, to present a solemn and urgent warning to every Member of Parliament and Senator, and indeed to every Canadian:

The fundamental human right of privacy in Canada is under assault as never before. Unless the Government of Canada is quickly dissuaded from its present course by Parliamentary action and public insistence, we are on a path that may well lead to the permanent loss not only of privacy rights that we take for granted but also of important elements of freedom as we now know it.

We face this risk because of the implications, both individual and cumulative, of a series of initiatives that the Government has mounted or is actively moving toward. These initiatives are set against the backdrop of September 11, and anti-terrorism is their purported rationale. But the aspects that present the greatest threat to privacy either have nothing at all to do with anti-terrorism, or they present no credible promise of effectively enhancing security.

The Government is, quite simply, using September 11 as an excuse for new collections and uses of personal information about all of us Canadians that cannot be justified by the requirements of anti-terrorism and that, indeed, have no place in a free and democratic society.

Why doesn’t the United States have a Privacy Commissioner?

And this:

A popular response is: “If you have nothing to hide, you have nothing to fear.”

By that reasoning, of course, we shouldn’t mind if the police were free to come into our homes at any time just to look around, if all our telephone conversations were monitored, if all our mail were read, if all the protections developed over centuries were swept away. It’s only a difference of degree from the intrusions already being implemented or considered.

The truth is that we all do have something to hide, not because it’s criminal or even shameful, but simply because it’s private. We carefully calibrate what we reveal about ourselves to others. Most of us are only willing to have a few things known about us by a stranger, more by an acquaintance, and the most by a very close friend or a romantic partner. The right not to be known against our will – indeed, the right to be anonymous except when we choose to identify ourselves – is at the very core of human dignity, autonomy and freedom.

If we allow the state to sweep away the normal walls of privacy that protect the details of our lives, we will consign ourselves psychologically to living in a fishbowl. Even if we suffered no other specific harm as a result, that alone would profoundly change how we feel. Anyone who has lived in a totalitarian society can attest that what often felt most oppressive was precisely the lack of privacy.

Great stuff.

EDITED TO ADD (7/6): That’s the 2001-2002 report. This is the latest report.

Posted on July 6, 2006 at 7:49 AM • View Comments

Wiretappers' Conference

I can’t believe I forgot to blog this great article about the communications intercept trade show in DC earlier this month:

“You really need to educate yourself,” he insisted. “Do you think this stuff doesn’t happen in the West? Let me tell you something. I sell this equipment all over the world, especially in the Middle East. I deal with buyers from Qatar, and I get more concern about proper legal procedure from them than I get in the USA.”

Read the whole thing.

Posted on June 29, 2006 at 1:43 PM • View Comments

Applying CALEA to VoIP

“Security Implications of Applying the Communications Assistance to Law Enforcement Act to Voice over IP,” paper by Steve Bellovin, Matt Blaze, Ernie Brickell, Clint Brooks, Vint Cerf, Whit Diffie, Susan Landau, Jon Peterson, and John Treichler.

Executive Summary

For many people, Voice over Internet Protocol (VoIP) looks like a nimble way of using a computer to make phone calls. Download the software, pick an identifier and then wherever there is an Internet connection, you can make a phone call. From this perspective, it makes perfect sense that anything that can be done with a telephone, including the graceful accommodation of wiretapping, should be able to be done readily with VoIP as well.

The FCC has issued an order for all “interconnected” and all broadband access VoIP services to comply with Communications Assistance for Law Enforcement Act (CALEA)—without specific regulations on what compliance would mean. The FBI has suggested that CALEA should apply to all forms of VoIP, regardless of the technology involved in the VoIP implementation.

Intercept against a VoIP call made from a fixed location with a fixed IP address directly to a big internet provider’s access router is equivalent to wiretapping a normal phone call, and classical PSTN-style CALEA concepts can be applied directly. In fact, these intercept capabilities can be exactly the same in the VoIP case if the ISP properly secures its infrastructure and wiretap control process as the PSTN’s central offices are assumed to do.

However, the network architectures of the Internet and the Public Switched Telephone Network (PSTN) are substantially different, and these differences lead to security risks in applying the CALEA to VoIP. VoIP, like most Internet communications, are communications for a mobile environment. The feasibility of applying CALEA to more decentralized VoIP services is quite problematic. Neither the manageability of such a wiretapping regime nor whether it can be made secure against subversion seem clear. The real danger is that a CALEA-type regimen is likely to introduce serious vulnerabilities through its “architected security breach.”

Potential problems include the difficulty of determining where the traffic is coming from (the VoIP provider enables the connection but may not provide the services for the actual conversation), the difficulty of ensuring safe transport of the signals to the law-enforcement facility, the risk of introducing new vulnerabilities into Internet communications, and the difficulty of ensuring proper minimization. VOIP implementations vary substantially across the Internet making it impossible to implement CALEA uniformly. Mobility and the ease of creating new identities on the Internet exacerbate the problem.

Building a comprehensive VoIP intercept capability into the Internet appears to require the cooperation of a very large portion of the routing infrastructure, and the fact that packets are carrying voice is largely irrelevant. Indeed, most of the provisions of the wiretap law do not distinguish among different types of electronic communications. Currently the FBI is focused on applying CALEA’s design mandates to VoIP, but there is nothing in wiretapping law that would argue against the extension of intercept design mandates to all types of Internet communications. Indeed, the changes necessary to meet CALEA requirements for VoIP would likely have to be implemented in a way that covered all forms of Internet communication.

In order to extend authorized interception much beyond the easy scenario, it is necessary either to eliminate the flexibility that Internet communications allow, or else introduce serious security risks to domestic VoIP implementations. The former would have significant negative effects on U.S. ability to innovate, while the latter is simply dangerous. The current FBI and FCC direction on CALEA applied to VoIP carries great risks.

Posted on June 28, 2006 at 12:01 PM • View Comments

Greek Wiretapping Scandal

Back in February, I wrote about a major wiretapping scandal in Greece. The Wall Street Journal has a really interesting article (link only good for a week, unfortunately) about it:

Behind the bugging operation were two pieces of sophisticated software, according to Ericsson. One was Ericsson’s own, some basic elements of which came as a preinstalled feature of the network equipment. When enabled, the feature can be used for lawful interception by government authorities, which has become increasingly common since the Sept. 11 terror attacks. But to use the interception feature, operators like Vodafone would need to pay Ericsson millions of dollars to purchase the additional hardware, software and passwords that are required to activate it. Both companies say Vodafone hadn’t done that in Greece at the time.

The second element was the rogue software that the eavesdroppers implanted in parts of Vodafone’s network to achieve two things: activate the Ericsson-made interception feature and at the same time hide all traces that the feature was in use. Ericsson, which analyzed the software in conjunction with Greece’s independent telecom watchdog, says it didn’t design, develop or install the rogue software.

The software allowed the cellphone calls of the targeted individuals to be monitored via 14 prepaid cellphones, according to the government officials and telecom experts probing the matter. They say when calls to or from one of the more than 100 targeted phones were made, the rogue software enabled one of the interceptor phones to be connected also.

The interceptor phones likely enabled conversations to be secretly recorded elsewhere, the government said during a February 2006 news conference. At least some of the prepaid cellphones were activated between June and August 2004. Such cellphones, particularly when paid for in cash, typically are harder to trace than those acquired with a monthly subscription plan.

Vodafone claims it didn’t know that even the basic elements of the legal interception software were included in the equipment it bought. Ericsson never informed the service provider’s top managers in Greece that the features were included nor was there a “special briefing” to the relevant technical division, according to a Vodafone statement in March.

But Ericsson’s top executive in Greece, Bill Zikou, claimed during parliamentary-committee testimony that his company had informed Vodafone about the feature via its sales force and instruction manuals.

Vodafone and Ericsson discovered something was amiss in late January 2005 when some Greek cellphone users started complaining about problems sending text messages. Vodafone asked Ericsson to look into the issue. Ericsson’s technicians spent several weeks trying to figure out the problem, with help from the equipment maker’s technical experts at its headquarters in Sweden. In early March of that year, Ericsson’s technicians told Vodafone’s technology director in Greece of their unusual discovery about the cause of the problems: software that appeared to be capable of illegally monitoring calls. It’s unclear exactly how the rogue software caused the text-messaging problem.

Ericsson confirmed the software was able to monitor calls, and Vodafone soon discovered that the targeted phones included those used by some of the country’s most important officials. On March 8, Mr. Koronias ordered that the illegal bugging program be shut down, in a move he has said was made to protect the privacy of its customers. He called the prime minister’s office the next evening.

The head of Greece’s intelligence service, Ioannis Korantis, said in testimony before the parliamentary committee last month that Vodafone’s disabling of the software before authorities could investigate hampered their efforts. “From the moment that the software was shut down, the string broke that could have lead us to who was behind this,” he said. Separately, he distanced his own agency from the bugging effort, saying it didn’t have the technical know-how to effectively monitor cellphone calls.

Posted on June 22, 2006 at 1:25 PM • View Comments

NSA Combing Through MySpace

No surprise.

New Scientist has discovered that Pentagon’s National Security Agency, which specialises in eavesdropping and code-breaking, is funding research into the mass harvesting of the information that people post about themselves on social networks. And it could harness advances in internet technology – specifically the forthcoming “semantic web” championed by the web standards organisation W3C – to combine data from social networking websites with details such as banking, retail and property records, allowing the NSA to build extensive, all-embracing personal profiles of individuals.

Posted on June 15, 2006 at 6:13 AM • View Comments

1 … 81 82 83 84 85 … 93

Sidebar photo of Bruce Schneier by Joe MacInnis.