Schneier on Security
A blog covering security and security technology.
« DEITYBOUNCE: NSA Exploit of the Day |
| IRONCHEF: NSA Exploit of the Day »
January 3, 2014
Cost/Benefit Analysis of NSA's 215 Metadata Collection Program
It has amazed me that the NSA doesn't seem to do any cost/benefit analyses on any of its surveillance programs. This seems particularly important for bulk surveillance programs, as they have significant costs aside from the obvious monetary costs. In this paper, John Mueller and Mark G. Stewart have done the analysis on one of these programs. Worth reading.
Posted on January 3, 2014 at 6:10 AM
• 36 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
I suspect their argument would be that it is impossible to predict how valuable a piece of hoovered up information *could* be for future use.
Why should NSA bother to do any cost/benefit analysis ? In any case, they are funded by the tax payer and have no oversight. Every year the US government keeps increasing their funding without justification or debate. On the other hand, NASA's budget gets cut. Ironic ...
Seriously. The surveillance industrial complex is so used to having blank cheques thrown at them and immunity from publishing budgets or expenses or any kind of rudimentary accounting. Why in the hell would they bother with cost/benefit? They are junkies with an endless supply of heroin. They are beyond caring.
Bruce, I think you are selling short some of the brightest political minds that exist, from the IC leadership teams to Executive Branch staffers, by saying they aren't performing a CBA. The salient problem I see with the paper you cite is that the authors are not able to see the benefits of the program in its totality, only anecdotally from isolated open sources. Further, the true maximum benefit of the program would be to actually prevent just a single attack on the scale of 9/11 or larger. In this way, the true benefit of the program could never cost enough, if it were to go on to save thousands of American lives. Just because the 215 program hasn't prevented another 9/11 does NOT mean it won't be successful at preventing one tomorrow, or that it has not been successful in filling CT analyst notebook's with datapoints which did not exist without it, allowing us to prevent terrorism catastrophes with nexuses overseas, but with some US component or person involved.
To assert that they don't do cost-benefit analysis is naive. It suggests they are either incompetent (lacking basic reasoning and collaboration skills to prioritize their limited resources) or that they do not have limited resources. Both are silly propositions. If anything, the disclosure of information has shown a very competent agency. The argument and evidence for limited resources is perhaps less obvious but no less reasonable.
I appreciate that many have the perception that the agency doesn't have limited resources. However, I think this is much more myth than reality. It is easy for those who don't understand to assume a newly exposed and vast capability must not be limited by resources. But there is little appreciation for the accumulation of capabilities over decades. The movies purport the myth of unlimited resources as well further distorting the perceptions of the average joe. There are no unlimited resources.
yeah, to say nothing of the fact that this "analysis" (read: diatribe) utterly fails to rigorously assess the only cost that matters in this case, namely to what extent a 4th amendment interest is invoked in the collection and analysis of the data respectively. it would be more interesting to determine precisely what volume of data and what compexity of search algorithm is required for a search of telephony metadata to actually generate the kind of emergent information thay the ACLU et al claims is invasive.
as for the benefit piece, we all know that intelligence work involves putting together tiny pieces of information together more often that it involves catching the big fish. this is especially true when working with metadata which by definition is not information-rich until combined with other metadata and/or content. explaining how information from 215 is used would therefore invoke other intelligence equities which the government may wish to protect. in any case it is incorrect to approach the 215 information in vacuo.
But the programs are so cool! And no one is making them stop! Who cares how much they cost????
I think there has to be a back door benefit. Are the Republicans getting dirt on Democratic candidates (or contributors), or vice versa? Maybe we should ask David Petraeus about the potential consequences of this sort of data collection.
The NSA's share of the "black budget" is second only behind the CIA, at around $10 billion dollars.
A cost/benefit analysis is only important if you care about efficiency. Efficiency is only a factor if resources are scarce. $10 billion is about the size of the government of Yemen's budget. For this one agency, that's quite a bit.
One of the main issues in the Iran/Contra affair was the fact that the CIA found its own source of financing after Congress cut them off -- mainly, running guns and drugs, and using the mafia to launder the money.
Why wouldn't the NSA have something similar going on? If they're lying to Congress, they probably have something to hide. Not to suggest the NSA is laundering money, but given that they seem more interested in exploiting critical infrastructure than securing it, it's plausible to suppose they're engaged in, at the least, industrial or financial espionage.
If you're not familiar with the US "intelligence" agencies' role in industrial espionage, the European Parliament report on Echelon offers some interesting insights:
See, specifically, section 10.
Further, note that the NSA is the creation of executive orders, not of Congress. Congress is the one with the money. If the NSA is not accountable to Congress, why would they need to justify their budget?
Congress's National Security Agency Act of 1959 is noteworthy not insofar as it enumerates limitations for the agency, but is a litany of exemptions to reporting and financial compensation requirements.
Congress doesn't want to know:
"EC.6.50 U.S.C. 3605 (a) Except as provided in subsection
(b) of this section, nothing in this Act or any other law (including, but not limited to, the first section and section 2 of the Act of August 28, 1935 (5 U.S.C. 654) 2) shall be construed to require the dis-closure of the organization or any function of the National Security Agency, of any information with respect to the activities thereof, or of the names, titles, salaries, or number of the persons employed by such agency. "
Congress doesn't want to know:
"(b)(1) In order to maintain necessary capability in foreign language skills and related abilities needed by the National Security Agency, the Director, without regard to subchapter IV of chapter 55 of title 5, United States Code, may provide special monetary or other incentives to encourage civilian cryptologic personnel of the Agency to acquire or retain proficiency in foreign languages or special related abilities needed by the Agency. "
Congress doesn't want to know:
"(f) The Director may waive the applicability of any provision of chapter 41 of title 5, United States Code, to any provision of this section if he finds that such waiver is important to the performance of cryptologic functions."
Congress doesn't want to know:
"i) The Director of the National Security Agency, on behalf of the Secretary of Defense, may, without regard to section 4109(a)(2)(B) of title 5, United States Code, pay travel, transportation, storage, and subsistence expenses under chapter 57 of such title to civilian and military personnel of the Department of Defense who are assigned to duty outside the United States for a period of one year or longer which involves cryptologic training, language training, or related disciplines."
The very notion that NSA is an intelligence agency is naive. Take, for example, the words of former CIA agent Ralph McGehee:
"The CIA is not now nor has it ever been a central intelligence agency. It is the covert action arm of the President's foreign policy advisers. In that capacity it overthrows or supports foreign governments while reporting "intelligence" justifying those activities. It shapes its intelligence, even in such critical areas as Soviet nuclear weapon capability, to support presidential policy. Disinformation is a large part of its covert action responsibility, and the American people are the primary target audience of its lies"
It is the job of a spy to lie. Spy agencies will lie about what they do. These types of institutions are not compatible with a open society.
“A cost/benefit analysis is only important if you care about efficiency. Efficiency is only a factor if resources are scarce. $10 billion is about the size of the government of Yemen's budget. For this one agency, that's quite a bit.”
Good links. Why hasn’t the Administrator reduced the NSA’s budget in these tight economic times? What is stopping the President from doing so?
[Questions about NSA surveillance program]
"It seems likely that “on net” (as the President puts it) the highly-controversial 215 program could also safely be retired for “operational and resource reasons” with little or no negative consequences to security as recommended in December 2013 by the President’s Review Group on Intelligence and Communications Technologies. If the 215 program has done little (and probably nothing) special to prevent or disrupt terrorist attacks in the United States, and if we are now having a healthy debate about the NSA programs, it seems reasonable to suggest that, even without full information about how the program costs, we are paying too much..."
This is a good report. The government is strapped for cash. Well over 50 percent of the voters dislike the bulk data collection program (I would guess about 70% of voting population hates it and would see its demise as a victory for Obama). The rest of the world would applaud it's demise.
Why hasn’t President Obama acted on de-fund 215 and other bulk collection programs? Is there some ulterior motive keep such a wasteful and politically unpopular program a float?
Is it perfectly reasonable that in the US credit bureaus know what credit and bank accounts an individual has? Millions agree to provide this information as terms of agreement for opening a bank or credit account. Investigators can ask a credit bureau for information about a suspect without a warrant. They can ask credit card companies for info without a warrant. They can pull credit usage details again without a warrant.
Is it not reasonable that an investigator can ask a telecomm provider about the status of an account? Isn't a phone number granted as part of an agreement between a person or business and the telecomm company? How is this really different than a credit card?
This canard that people are voluntarily opening bank accounts, using phones and carrying computers across borders is an absolute lie. In our current society, the pursuit of happiness -- and anything above a poverty income -- requires all of these things.
The idea that we should be willing to give up our rights to avoid any risk is cowardly and absurd, since we would reduce our risk much more significantly by not pursuing a foreign policy that is guaranteed to create enemies. For good reason. How would you feel if your wedding party was blown away by a drone? Your child imprisoned without a trial? etc. etc.
It should be clear that our foreign policy is in fact DESIGNED to create terrorists by the people in western "democracies" who benefit from terrorism: Totalitarians who want to limit our freedoms and control our society, businesses that profit from war and "national security", and politicians who are controlled by these, either through blackmail or money or promises of wealth as soon as they leave "public service".
To call what the politicians today do "public service" calls to mind the Twilight Zone (or whatever) where aliens had a book about how they could serve humans -- that turned out to be a cookbook.
You can't put a price on tyranny.
We want to live in a cloud city. We want to monitor the planet without being monitored ourselves. I want that, but only if I can live in the cloud. If I have to live on the planet, I want shelter from the cloud.
The NSA is just a tool for keeping the cloud afloat. Like most tools, what it does depends on who holds it and what they want to do.
Absence of evidence is not evidence of absence. However, most organizations under the gun and in the spotlight would be pulling out all the stops to prove their value and virtue. The NSA still isn't working real hard at looking good - despite having been in the public eye for >6 months now. Concrete evidence of their effectiveness is not being made public.
This is strong evidence that they continue to believe they are immune from trivialities like oversight and accountability. Impressing or mollifying elected representatives seems to be a low priority. I suspect that when the security elite speak in private, expressions of searing contempt for us all are The norm. The average citizen (and his slimey representative) is viewed as too undisciplined, ignorant and feckless to have any say in how his tax money is being spent on surveillance.
If you are unaccountable, have a secret unlimited budget and a mission sent directly from God to save the world, anything is possible.
I like the part about "Why is this a secret, how is making the program public hurting your efforts?" "That's a secret." It seems to me like the best way to subvert democracy is to not allow the public to know what's going on inside the government.
It seems like they do have a cost benefit analysis, it's just that benefit is measured in volumes of data, not quality of intelligence.
To be perfectly honest, I am not that scared of terrorists, but it seems to me like intelligence assets, informants inside the organizations, and targeted spying is the most effective way to combat terrorism. Of course, that is second to ending imperialsim and working to build allies as allies instead of exploitable resources, working to end poverty worldwide, improving the quality of life within our own borders, and showing the world that the US is a country that brings prosperity to its allies, not instability and depletion of natural resources. The best defense is not making enemies everywhere you go.
Maybe they just didn't think it was necessary due to them being so into the whole information is power idea they thought it was self explanatory.
""The idea that we should be willing to give up our rights to avoid any risk is cowardly and absurd, since we would reduce our risk much more significantly by not pursuing a foreign policy that is guaranteed to create enemies.""
Youtube videos well worth listening to.
Noam Chomsky: Who Owns the World?
Noam Chomsky "A People Centered Society"
On a side note: $1 billion additional spending on medical research would have a much greater impact on life expectancy than preventing one 9/11 per year.
""Why hasn’t President Obama acted on de-fund 215 and other bulk collection programs? Is there some ulterior motive keep such a wasteful and politically unpopular program a float? ""
The political will of the people hasn't been there to allow him to do it.
He doesn't need the will of the people to do it; we had nothing but speculation and rumors until last year, but he's the head of the executive branch, ultimately in charge of the NSA. I don't think it's necessarily malice, I just think that the NSA is very good at convincing people that more information means better security.
The people who are capable of getting elected to major offices in the United States tend to be people who are not very strong in their convicitions. They are people who are very good at tweaking their views for each crowd. In other words, they are malleable, they have to be to win the pageant. On top of that, they tend to be people who generally support the status quo (fear of change is strong among voters), and they tend to be people who are very concerned about the prosperity of their parties, and the military industrial complex is both a major donor and has been the status quo since the cold war.
Susu Rosak says:
> Further, the true maximum benefit of the program
> would be to actually prevent just a single attack on
> the scale of 9/11 or larger. In this way, the true
> benefit of the program could never cost enough, if it
> were to go on to save thousands of American lives.
So to save a few thousand lives, it is acceptable to spend *unlimited* resources? For round numbers, let's say that the program will save 10,000 lives per year.
Is it worth a billion dollars per year? That's $100,000 per life saved, and around $3 per person in the US. Probably a reasonable deal.
Ten billion? A million dollars per life saved, and $30 per person in the US?
A trillion? A hundred million dollars per life saved, and $3000 per person in the US?
How about $16 trillion per year - $1.6 billion per life saved, and roughly the entire economic output of the US? Every working moment of every person's life, devoted to saving those 10,000 lives? What would they eat?
Life is valuable, but its value is not infinite.
> On a side note: $1 billion additional spending on medical research would have a much greater impact on life expectancy than preventing one 9/11 per year.
Actually, the productivity of the US healthcare system has been in steady decline since the vaccine. Since then, most of modern medicine has been devoted to addressing problems of industrial civilization, such as poor diet, sedentary lifestyle, and environmental pollution. Healthcare spending is subject to diminishing returns just like anything else:
Life in a hunter-gatherer society is not "nasty, brutish, solitary and short." Rather, a typical hunter-gatherer might expect a life expectancy on par with Renaissance Europe. A reasonable portion of the population can expect to remain active and healthy into their 70's, with a social support network to care for them as the age.
Basically, modern medicine has benefited from relatively few innovations: sanitation and hygiene (i.e., Ignaz Semmelweis in the mid 1800's), anesthetics and analgesics (patients don't die of shock at seeing their own leg cut off anymore), antibiotics (developed for around $20,000 of basic science research), and the vaccine. And that's it. Take away painkillers, take away antibiotics, and modern surgery is no longer possible.
If you're going to use "one 9/11" as a metric for how to save lives, think of this: as many people die in car accidents EVERY MONTH as died on that one day 12 years ago. If you want to spend $1 billion to save lives, regional mass transit is the way to go. Fewer drivers = fewer traffic fatalities.
An open letter to the National Security Agency (uh, you might want to read the wikipedia entry on "Open Letters" your excellency Mr. Alexander).
Where to begin...the damage wrought by National Security Agency (NSA) on one small company.
1.) The re-alignment of company resources to address what can only be described as the need to maintain data/information integrity based on risks to our business by government(s) agency.
2.) Modifications to short and long-term partnerships and development efforts based on issues surrounding the propriety of data/information/research and the failure of systems to be applicable to these objectives.
3.) Significant time/effort spent addressing colleagues, customers, clients, personnel, and even friends about the concerns and risks that would become more apparent over time, and, the legal implications with respect to diligence and propriety.
4.) Redefine internal/external operations models to directly address the outdated--rooted in the 1940's--compartmentalized and disjointed business operations models and embrace an "Open Business" model. This, with the need to maintain and improve IA/QA and business operations structure has been significant.
5.) Loss of project(s) and the inability to address existing projects sufficiently due to the aforementioned issues.
The National Security Agency, by its actions and policies, has directly injured our company and the people that make what we do interesting and relevant. Damage of this type was not considered possible in the recent past--but my what a difference a little knowledge makes.
THE NSA can go and f' itself.
Anyone who has the displeasure of working for the government in any substantial capacity then also having private sector experience knows cost-benefit analysis is about as familiar to the government as scuba diving is familiar to a donkey. There are no economic incentives for government agencies. "you don't use it, you lose it" is the budget incentive.
In fact, not only has the US secret intelligence agencies and the DoD broken every conceivable amendment to the constitution, but they may very well have dealt a death blow to the US tech sector. BILLIONS of revenues will be lost. Why would any rational customer buy US hardware software or services that are potentially hijacked?
"But they haven't done anything illegal" the deluded ones say. Let me break this down:
1st Amendment - National Security Letters, blatant threats to Journalists, such as: Chris Hedges, Glenn Greenwald, Fox news reporters, NYT reporters, Laura Poitras, etc.
2nd Amendment - Nothing public as of yet, but I'm sure there's something
3rd Amendment - Installing malware on innocent people's machines close to your target in order to do injection and monitoring certainly violates this. See Jacob Applebaum's 30c3 part 2 video.
4th Amendment - Let's spell this out for the dim ones about us:
"The right of the people to be secure in their persons, houses, papers and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the person or things to be seized."
The US government has actually found itself guilty of violating this amendment en masse! The NSA captures and stores all information they can get their hands on on everyone in the world including US citizens, with no judicial review, or congressional review, or really any review at all. However the President's administration has blocked the ruling from being made public. The administration has also asked for emergency injunctions in response to Chris Hedges suit against the NDAA indefinite detention and EXECUTION without trial of US citizens. But he's just a swell guy, just doesn't have the support necessary. BULL.
The 5th, 6t, 7th, 8th, 9th, and 10th have all been eviscerated by the government likely with involvement here and there from the NSA. But it's safe to say these amendments have been put to use as toilet tissue by several administrations including this one.
But it get's better, not only are they spying on every person in the world including US citizens. Combining the NDAA section 1021, indefinite detention, appointment of corrupt, dirty federal judges (that they can control), funding and arming Al Qaeda, illegal spying, etc. they are quite literally making war against the American populace, and the world. There is a special name for that kind of criminality: TREASON
Treason against the United States, shall consist only in levying War against them, or in adhering to their Enemies, giving them Aid and Comfort. No Person shall be convicted of Treason unless on the Testimony of two Witnesses to the same overt Act, or on Confession in open Court.
It's about power. The NSA, FBI and the CIA have attained unprecedented secret power unanswerable to anyone including congress. Extreme power is always dangerous, extreme power with no oversight is a guaranteed recipe for disaster and mass murder at the hands of the state. The democratic republic is nearly dead, and will be very difficult to get back. The NSA is merely the eyes of the monster. Just wait until the teeth and claws come out.
I was just using 9/11 as an opportunity/cost point. There are many things that could be spent on that would save as many or more lives a year.
BTW: Mass transit systems, for the scale needed to reach 9/11 death reduction rates, will easily have a multi-billion dollar price tag or even greater.
On Antibiotics. We are loosing the use of many of them to multiple drug resistant organisms, and there hasn't been the basic research needed to replace them with new ones at anywhere close to the rate that they are becoming ineffective. The research needed to find them hasn't been done by industry because the payoff is often decades out and not justifiable.
@clockmaker, I think a candle, in the dark and stormy night you write from, is that the government is saddled with its own bureaucracy. Sure they trample our rights to vacuum up all our metadata in a desperate hunt for "terrists." Let's hope they keep their focus on that. The terrists aren't going away, and the spooks are willing to ignore local crime in order to chase the shiny objects. It could be worse.
Megalomania doesn't do rational.
We're losing the use of antibiotics because they're over-used and used carelessly.
80% of antibiotics used in the US are given to livestock prophetically. This is starting to change, but slowly.
Factory farms keep animals in stressful, crowded, and unhygienic conditions that compromise the animals' immune systems and foster the spread of disease. Furthermore, these animals -- especially large livestock -- are fed high starch diets because there simply isn't enough pasture land in the US for all the cheap beef Americans want to be fed on pasture. These high starch and grain based diets allow harmful bacteria to ferment in the gut, which are then exposed to antibiotics, leading to resistance. These antibiotics and drug resistant bacteria enter the human food supply not so much through slaughtered meat (except in the case of chicken), but through the manure spread of vegetables, and through agricultural runoff.
If you eat meat, you're a big part of the problem. If Americans ate meat three times a week instead of three times a day, it would keep as much carbon out of the atmosphere as driving a hybrid electric vehicle (it takes 9x the energy to produce a pound of meat protein compared to a pound of vegetable protein), cut down on irresponsible antibiotics use, and probably increase your health, lowering healthcare costs.
As far as the opportunity cost of spending $1 billion on mass transit, you have to average the "life savings" over a 12 year period. Depending on where in the country this money is spent, this could have a significant impact.
But you're right, I may have overstated my case. Maybe some combination of homeless shelters, food programs, clothing drives, drug treatment, and gun buy-back programs would have more impact for the money.
prophetically = prophylactically
Preventable Nosocomial infections kill more by a margin of 10,000 to 1. So do many other preventable causes of death. And on and on. The automated surveillance state robs society of the funds necessary to improve our quality of live by leaching funds from thousands of important and needed works. The occupy movement understood this but it was a fringe movement, only 0.1 percent Americans participated. The FBI, NSA and Homeland Security worked together to arrest nearly 8,000 and squash it before it spread. The tea parties had a chance and also got marginalized. When surveillance is total and uses all available means to dismantle these populist groups how is a nationally networked mass movement popular enough to transform the nation possible when they infiltrate and destroy these movements early and from within?
1) Re: Cost-Benefit Analyses (Efficiency) in Government Budgets:
@ The concept in general:
My old boss had an excellent PPT intro to Congressional budgets. IIRC, a much longer and crappier version of it can be found at Kaiser ACA site, but it's focused exclusively on how the recent Congressional showdown and ACA impacts it. It used to be up on the HLS library site back in the day.
Basically, the budget can be broken into two parts: military spending and discretionary spending. Military spending is mandatory and abides by a completely different set of rules. I have no idea what they are because I never dealt with them. The librarians there may be able to help if it's not classified. Discretionary spending is a fixed pie and is handled after military spending, which is why it turns into such a blood bath.
The first chunk of discretionary spending is obligated to go to Social Security/Medicare. Everything else is "discretionary-discretionary." (e.g., USDA, CDC, etc.).
This is part of the reason why I saw the bluest of blue state LAs -- and I mean people who I knew were quite a bit to the left of Jon Stewart, just like their constituents -- twist themselves into mobius knots to try to come up arguments about how their food stamp/birth control pill/pure STEM research/etc. proposals were really a "terrorism" issue. If it's a "terrorism" issue, it's a military issue, which means it has to be funded.
Um, what division are you in? Mine required them for everything. Some CBAs are total b.s., some aren't, and it's hard to tell which is which. They're still mandatory.
Bingo. Almost anything the government could spend money on would save move lives than the "War on Terror"
@ Anonymous Coward (ex-LA variety)
You kind of have things backwards. Medicare and Social Security are most definitely NOT discretionary spending.
And there's nothing inherently "military" about terrorism issues. Most domestic terrorism cases are treated as criminal matters.
Entitlement programs are not part of the normal budgeting process because they are paid for out of a trust fund. They are NOT part of the deficit at all.
The trust fund pays benefits to current retirees and is replenished by current workers. The spending shortfalls associated with these entitlement programs relate largely to the fact that people are now living longer, or, more specifically, people are accumulating increasing medical expenses towards the end of their lives.
The funding shortfall for entitlement spending can be eliminated rather simply. Right now, only incomes up to about $110,000 are taxed for entitlement programs. Raising that limit would eliminate the problem.
Congress is aware of this, but they don't act:
"Although the legislated Social Security payroll tax rate is 12.4%, the average Social Security payroll tax is slightly progressive throughout the bottom 80% of the income distribution in that lower-income families pay a lower proportion of income in payroll taxes than higher-income families. At the higher-income levels—the top 20%—the payroll tax is regressive in that the proportion of income paid in payroll taxes falls as income rises. The richest 1% of American families pay a smaller proportion of their income in payroll taxes than the poorest 20% of families."
"Four policy options, which raise the payroll tax base, are examined; two of the policies also provide tax relief to low- and middle-income workers. Each of the three policies reduces the regressivity of the payroll tax at the upper end of the income distribution. Currently, less than 10% of families contain a worker earning more than the maximum taxable limit. Consequently, over 90% of families would be unaffected by increasing the maximum taxable limit. And if this change were combined with a payroll tax rate reduction, over 90% of families would pay lower payroll taxes."
@Anonymous Coward (ex-LA variety)
Of course they're required, just as doing a METT-TC analysis - Mission, Enemy, Terrain, Troops, Time available, Civilian Considerations is required for any operation. That doesn't mean that they've thought it out or are remotely correct. They have no fundamental basis for doing a CBA. Do you think murdering hundreds of children fits into the COIN (counterinsurgency) doctrine of winning hearts and minds?
You can put a heifer in front of a piano, but that doesn't mean she will play Bach.
Schneier.com is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc.