AT&T's 1.9-Trillion-Call Database

This whole article is worth reading, but I found this tidbit particularly interesting:

He was alluding to databases maintained at an AT&T data center in Kansas, which now contain electronic records of 1.92 trillion telephone calls, going back decades. The Electronic Frontier Foundation, a digital-rights advocacy group, has asserted in a lawsuit that the AT&T Daytona system, a giant storehouse of calling records and Internet message routing information, was the foundation of the N.S.A.’s effort to mine telephone records without a warrant.

An AT&T spokeswoman said the company would not comment on the claim, or generally on matters of national security or customer privacy.

But the mining of the databases in other law enforcement investigations is well established, with documented results. One application of the database technology, called Security Call Analysis and Monitoring Platform, or Scamp, offers access to about nine weeks of calling information. It currently handles about 70,000 queries a month from fraud and law enforcement investigators, according to AT&T documents.

A former AT&T official who had detailed knowledge of the call-record database said the Daytona system takes great care to make certain that anyone using the database – whether AT&T employee or law enforcement official with a subpoena – sees only information he or she is authorized to see, and that an audit trail keeps track of all users. Such information is frequently used to build models of suspects’ social networks.

The official, speaking on condition of anonymity because he was discussing sensitive corporate matters, said every telephone call generated a record: number called, time of call, duration of call, billing category and other details. While the database does not contain such billing data as names, addresses and credit card numbers, those records are in a linked database that can be tapped by authorized users.

New calls are entered into the database immediately after they end, the official said, adding, “I would characterize it as near real time.”

According to a current AT&T employee, whose identity is being withheld to avoid jeopardizing his job, the mining of the AT&T databases had a notable success in helping investigators find the perpetrators of what was known as the Moldovan porn scam.

In 1997 a shadowy group in Moldova, a former Soviet republic, was tricking Internet users by enticing them to a pornography Web site that would download a piece of software that disconnected the computer user from his local telephone line and redialed a costly 900 number in Moldova.

While another long-distance carrier simply cut off the entire nation of Moldova from its network, AT&T and the Moldovan authorities were able to mine the database to track the culprits.

Tags: , , , , , , , , , , , , , ,

Posted on March 3, 2006 at 11:45 AM • 47 Comments

Comments

roy • March 3, 2006 12:19 PM

Obviously, AT&T understood decades ago that information about people’s communication was a commodity that could be sold, resold, and sold again, without loss of product, without the customers’ knowledge, and against their customers’ will.

They say they have an audit trail, but I would bet my life they also have at least one means of access that bypasses the auditing mechanism.

jmr • March 3, 2006 12:44 PM

Re Moldova: But why bother “mining” the database? Couldn’t they just look at the number called by a single complainant?

This article seems to jump from “It’s okay to keep some data for a short period of time to investigate problems” to “It’s okay to keep all data forever so we can retroactively examine every aspect of anybody’s life” by starting with the bigger scenario (keep everything forever) and showing how the smaller scenario made things better for the world.

James Walker • March 3, 2006 1:51 PM

While this doesn’t seem like much (if anything) new, it does raise a couple of interesting issues.

Roy suggests AT&T is motivated by profit for maintaining this database; it could be that AT&T is required by the government to do so. Does anyone have hard info on this?

More to the point, this story is on the news radar because the EFF is taking legal action against AT&T. (The EFF claims AT&T gave the NSA “unfettered access” to the database.) I think it’s a very interesting element of the wiretapping scandal fallout. Think about it: if the EFF wins and AT&T has to pay real money, it could severely alter the willingness of corporations to comply with wiretapping requests.

How’s that for checks and balances?

Glauber Ribeiro • March 3, 2006 2:03 PM

It’s the good old law of unintended consequences: “the Daytona system takes great care to make certain that anyone using the database – whether AT&T employee or law enforcement official with a subpoena – sees only information he or she is authorized to see”

but now the government argues that they are authorized to see everything, based on their self-assigned war powers.

Sean • March 3, 2006 3:09 PM

“[T]he company would not comment on … matters of national security or customer privacy” …because we as a people have neither.

Realistic • March 3, 2006 9:35 PM

Re: national security, customer privacy

Never has been any and what you thought was there, was merely an illusion. Why do you think they call them spooks and spies; greedy, stupid politicians drunk on power; and evil enterprises. Moreover, there never will be any national security, or customer privacy. Get over it.

Jojo • March 4, 2006 2:34 AM

Remember – The original AT&T was brought by SBC, who adopted the AT&T name in place of SBC. So now the old SBC owns all this data. Maybe the value of this “asset” is one reason why SBC was willing to purchase a company that looked close to going out of business.

TJ • March 5, 2006 7:14 PM

Realistic wrote:
Moreover, there never will be any national security, or customer privacy. Get over it.

You’re so sure? I’m pretty sure millions of us won’t “get over it”. And “never” is a long time. There’s a difference between “realistic” and “submissive”.

Andrew • March 6, 2006 7:23 AM

I’m sure time and date stamps are kept along with the phone numbers. What would be interesting to see is a graph by date on the number of phone calls that happened each day. I would be interested in seeing the trend of calls on historical event days such as 9/11.

former bell employee • March 6, 2006 9:31 AM

FWIW, I don’t think they have records of strictly “local” calls.

Transparent Society • March 6, 2006 9:39 AM

Instead of just pretending that this information can be kept private, why don’t they just:

(1) notify customers when their call records are being searched, bought, or sold, and by whom, and

(2) make the database available to everybody, not just certain government employees.

If some cop or other government bureaucrat wants to snoop through my phone records, I want to know about it, and have the same ability to search throug their records. After all, they have nothing to hide, right?

Keith • March 6, 2006 9:54 AM

Imagine what Google could do with that database, figuring out relative relevance of different people and who is connected to whom the way they do with hyperlinks. Privacy issues aside, it would be an interesting experiment…

Alan • March 6, 2006 10:15 AM

All of you end-of-the-world types aren’t being skeptical enough. Think for a moment how much storage would be required to reposit “all call information” going back decades. The amount of rotating storage wouldn’t have been affordably available until relatively recently. Perhaps not even now. If all of the records were kept in cold storage, i.e. tape, then the sheer volume of of them and the lack easy access effectively makes them lost. No one would go through the trouble to investigate the indexes, compile the reconstruction requests and then (pray that the tapes are still readable) peruse the resulting database unless he or she had tremendous will, time, patience and money.

If someone wanted to argue that only international calls to certain countries and/or suspicious individuals were logged, I might buy that, but every call? No way. Didn’t happen.

Time to come back in off the ledge.

Peter • March 6, 2006 10:17 AM

Call records — who called who and when — are not private (as opposed to the contents of the call) and it’s perfectly legal to sell them or mine them for data. So you call Mexico five times a week, guess what? Soon you get a telemarketer phone call offering you cheaper phone rates to Mexico.

If you’re worried about it, you can call the phone company and have them stop using your data. You can also have them not record it at all, but that means your phone bill will no longer itemize your calls. If you care.

Railroad consultant • March 6, 2006 12:22 PM

People who obsess about the government invading their privacy assume they’re important enough that someone cares about what they’re saying.

Guess what? Nobody cares. If the idea of call records being kept makes you hysterical, disconnect your telephone.

shoobe01 • March 6, 2006 12:37 PM

“…then the sheer volume of of them and the lack easy access effectively makes them lost.”

I work for a telecom, and strongly agree. Seriously, I am not as paranoid as I could be regarding data mining because its practically impossible to get normal usage reports when you actually work for the company. Lots of data, badly stored, completely independent of other data, etc., etc.

Knucklehead • March 6, 2006 1:30 PM

Hopefully this will help some what.

It has been a very long time since I had to deal with the specifics of this sort of thing so I’m working from distant memory.

Whenever anyone makes a phone call an electronic record, a Call Detail Record (CDR) is made. It carries information essential to billing such as originating #, destination #, TOD, duration, etc. IIRC this is somewhere around 240 bytes (240 characters) of information. This information is not a recording of the actual phone calls – it is the details about the phone calls.

Service providers (telephone companies) obviously need this information to bill for their services but they also have statutory requirements that must be met. (See CALEA.) These statutes go a long way back – probably started in the 30’s or even much earlier.

My recollection of this is that the phone companies (and once upon a time Ma Bell was pretty much THE phone company) are required, by federal law, to maintain CDRs for 7 years. The regulations also, IIRC, had a “turnaround time” factor. So, for example, a duly authorized law enforcement agent with suitable subpoena or whatever the equivalent required by the particular statutes is, could demand, from any phone company, CDRs up to seven years old and the phone company was required to respond to the law enforcement demand within some time limit (72 hours rings a distant bell). So, for example, investigators could go to the phone company and demand the CDRs for, say, the year 1989 made from, or received by, thus and such telephone #(s).

This was massive amounts of data that could not be kept on short term storage (like disk drives). Phone companies built vast – and I mean vast – tape libraries and such of these CDRs. Today these libraries are CDs and such – enormous, hi-tech jukebox thingies with the sorts of database software to retrieve records and print out the reports demanded by law enforecment.

Hundreds of millions of phone calls are made each day in the US. This means hundreds of millions of CDRs are collected each and every day. It isn’t the least bit suprising that AT&T would have CDRs for two trillion calls. I’m too lazy to do the arithmetic but, offhand, that sounds like somewhere around 5 to 7 years of CDRs.

I don’t recall ever hearing that there was some requirement for detroying CDR records once the statutory requirement for maintaining them had passed. Even if there is no such requirement, however, the cost of maintaining CDR libraries would make indefinite storage implausible if not impossible. Just briefly considering the technological changes suggests that keeping CDR records for more than 10 years or so would be cost prohibitive. More than twenty years starts to look like a technologically impossible problem.

Tsymyn • March 6, 2006 1:48 PM

Source and destination phone numbers and a datestamp and length would only be about 40 bytes per record, maybe less depending on what data types they use. At that rate 1.9 trillion would be about 69 terabytes. A 7 terabyte Apple storage array (cheaper than Dell, HP, etc.) is $13k. So storing that much data would only be about $130,000, not remotely cost prohibitive for a company that has a multi-billion dollar bid in on a competitor.

Tsymyn • March 6, 2006 1:53 PM

Refiguring for Knucklehead’s recollection, you’d need about $780,000 for network storage plus maybe another half mil in supporting infrastructure to have immediate access to that much data if you bought the equipment today. Still well within a large company’s means. They spend more than that replacing office furniture. Once flourescent multilayer discs take off they’ll be able to back that puppy up on a dozen multilayer dvds. Heh. Of course, storage is exponetially cheaper and smaller now than it was two or three years ago.

m • March 6, 2006 2:04 PM

There is a serious scandal about the availability of the pen register (call description data) information being readily available for just about all phones. A hundred bucks or so will buy a months worth of calls to and from a particular number with data going back 20 years.

John from Americablog details his experience at buying 100 call records for General Wesley Clarks cell phone. So it is probably easier and cheaper for LEO to buy the records online than to bother with a warrant. You can read about it at:

http://americablog.blogspot.com/2006/01/americablog-just-bought-general-wesley.html

Congress has talked about making this a crime, but has done nothing so far that I know of.

Knucklehead • March 6, 2006 2:38 PM

Tsymyn,

There’s no point being symplistic. If you want to believe this AT&T DB is a “mere” (by today’s standards) 70TB built upon some Apple storage arrays and sitting in some basement somewhere at AT&T and that they built if for a couple hundred $K that’s your right.

You may been born recently but CDRs, and the many variations of them to meet changing network technology, weren’t.

Bill McGonigle • March 6, 2006 2:55 PM

Call me a heretic, but given that database and what’s known about the 2001 Al Qa’e’da attack on America, I’d sure be building graphs of the calls to and from known locations of the cell and doing social network analysis on those graphs.
Sure, there’s opportunity for abuse, sure it needs oversight, but, my goodness, there’s alot of information there to find the hidden actors. Now that this is out, they’ve probably all jumped to skype or similar, but it was smart when it was surreptitious.

Anonymous • March 6, 2006 3:37 PM

Disk drives didn’t reach 1GB capacity until circa 1980. When they did reach that they weighed hundreds of pounds, were slow, cost a fortune, and threw off a lot of heat.

As little as 10 or 12 years ago (try researching what technology was necessary even the statutory requirement of seven years ago) building a DB of this scale required complex farms of primary, secondary, and tertiary storage. The tape robots were the size of rooms. The tape libraries required thousands upon thousands of environmentally controlled square feet. Back in the mid-90s the statutory need, for seven years, was somewhere around 70TB. That has almost certainly close to doubled in the decade since.

CDRs got “aged” over time and migrated from primary storage (disk) to secondary storage (robotic tape libraries) to tertiary storage (warehoused archives).

Once you’ve looked at that, go review the accounting regulations to determine how many years it takes to depreciate the equipment. These things don’t get tossed and redone just because technology changed. Just because you can build it for a million today doesn’t mean you could do so seven years ago and fourteen years ago the problems and costs were even more immense.

Nobody compensates the telcos for this which is all the more reason they tried to keep the price down. And keep in mind that there was, probably still is, statutory requirements here which means the potential for large fines for non-compliance. If you’re going to put it on disk you’d better use RAID technology to mirror it which, essentially, doubles the disk costs you’ve run you’re little calculation on as well as the electricity and cooling costs. Those costs were never insignificant.

In the days before ubiquitous cell phones attached to nearly every man, woman, and child in the developed world (which was less than a decade ago) just one of the large long distance carriers produced a couple hundred million CDRs per day. VoIP and cell phones have played havoc with the legislation involved (which is why you can purchase people’s cell phone records but you can’t purchase their land line phone records). It has also vastly increased the call traffic that generates CDRs.

Back in 2000 US telephony network traffic was something like 250,000 terabytes per month. And data did not yet equal voice. Voice hasn’t been growing radically but data has.

This is not a “small” technology problem. It is a very large and expensive one.

Anonymous • March 6, 2006 3:44 PM

Bill,

I’m in the same camp you are. I have no big issue with datamining all the CDRs available. When somebody starts prosecuting ordinary Americans for something other than terrorist associated crimes with evidence mined from CDRs then it would be a different matter.

I’m having some difficulty understanding what it is about their CDRs that leads people to be so concerned about the government mining them.

Nealjking • March 6, 2006 5:19 PM

I think the concern is not about the content of the information but about the principle of what the Administration finds to be permissible to collect. It is not hard to believe that, whatever you think is a reasonable exercise of executive power, that someone will at some point stretch beyond that, and possibly try to create a further precedent. Regardless of one’s political leaning, it is likely that there will someday be an Administration that one doesn’t particularly trust. That Administration will inherit all the powers and precedents and tools that have been granted or allowed up to that point. So if you think you might ever have any doubt about the goodwill or integrity of the executive branch, it would behoove you to watch a stretched claim of need for personal information with a bit of a skeptical eye.

rosignol • March 7, 2006 4:05 AM

Roy suggests AT&T is motivated by profit for maintaining this database; it could be that AT&T is required by the government to do so. Does anyone have hard info on this?

Part of it is billing records (those records are accounting information, and likely subject to certain retention requirements), and others already mentioned CALEA legislation. So ya, AT&T is definitely legally obligated to maintain this information for a certain amount of time.

As far as the logistics of doing so are concerned, well, I expect there are a few supercomputers plugged into fairly large tape libraries in a nondescript building somewhere. I wouldn’t worry too much, so long as the building isn’t in Maryland.

Loompanic Attack • March 7, 2006 6:49 AM

What’s baffling is that anyone thinks their calls (FromNumber, ToNumber, StartTime, EndTime) haven’t always been recorded, or at least recordable. Why do you think mafiosi, spies, radicals, and dealers use pay phones, carnations, thumbtacks, and slang to communicate?

Knucklehead • March 7, 2006 10:09 AM

Nealjking,

This site doesn’t seem to take html instructions so I’m forced to resort to this:

You said:

That Administration will inherit all the powers and precedents and tools that have been granted or allowed up to that point. So if you think you might ever have any doubt about the goodwill or integrity of the executive branch, it would behoove you to watch a stretched claim of need for personal information with a bit of a skeptical eye.

I reply:

There are many examples throughout US history where the Executive branch has pushed it’s powers beyond what we would normally allow. Wartime conditions are ripe for this sort of thing (Lincoln, FDR, and others) have given us examples. Jefferson, and others, have provided us examples of Executive “stretch” in peacetime.

It happens. Sometimes the expansion, or some portion of it, becomes permanent but often it is rolled back when whatever crisis forces the issue finally passes. If we look at the (sometimes nominally) federalized departments that exist under the control of the Executive we see a whole lot that didn’t originally exist. Justice (FBI), ATF, IRS, and on and on and on. We accept the creation and operation of these things as necessary and, typically over time, discover the manners in which they are abused and do our best to rectify them.

CDRs are not exactly “personal information”. They are not a pure matter of “ain’t nobody’s business but my own”. When we sign up to have telephony services which we agree to pay for we also agree, either explicitly or implicitly, to accept the collection of data about our use of the service that will allow us to be billed for it. The provider of the service agrees to abide by the laws. There are, as I have pointed out, legal requirements for the legitimate purposes of law enforcement. There are also, as someone else pointed out, legal requirements for “accounting purposes”.

Data such as CDRs is not really “personal data” – it is data about us personally. For many years telephony was a regulated monopoly. It was, to a large extent, deregulated but it is still a very highly regulated area. The regulations are in place, ostensibly, to protect citizens and even corportations as they go about the legal conduct of their lives. The access that is allowed is allowed to gather some of the information needed to prosecute illegal conduct. A necessary (and in many ways imperfect) compromise that has been developed to suit our needs.

We the public are reasonably well protected from prosecution using evidence illegally acquired. There doesn’t seem to be any evidence that anyone is being prosecuted or persecuted through information gathered by datamining the vast CDR data warehouses. If and when some hint, wiff, or actual evidence of prosecution or persecution of “innocents” surfaces we’ll be in a position to counter that.

Pre-emptively denying the Executive reasonable tools to do what we need it to do is, well, silly. “Fersure somebody will someday abuse this power, therefore the power shouldn’t be granted” is not normally how we go about running our government. We have the fundamental protections of the Bill of Rights – the constitution – and then muddle through in the gray areas and deal with problems as they arise.

wm • March 7, 2006 11:08 AM

@Knucklehead:
‘They are not a pure matter of “ain’t nobody’s business but my own”. When we sign up to have telephony services which we agree to pay for we also agree, either explicitly or implicitly, to accept the collection of data about our use of the service that will allow us to be billed for it.’

Exactly. We agree to the collection of data for the purpose of billing us. Not “to be sold to any miscreant who wants it”.

Knucklehead:
‘”Fersure somebody will someday abuse this power, therefore the power shouldn’t be granted” is not normally how we go about running our government. We have the fundamental protections of the Bill of Rights – the constitution…’

Actually, that constitution was written under the explicit principle that “Fersure somebody will someday abuse this power, therefore the power shouldn’t be granted” [except for those powers that were absolutely necessary].

Pat Cahalan • March 7, 2006 11:38 AM

A great many of the posts on this thread assume a definition of “personal data” that is tailored for the person presenting the particular arguement.

Any data that is collected that can be used to identify me is personal data. If you’re assigning some sort of “weight” to some data that you’re not assigning to other data, you’re doing so based upon an assumption of use.

For example, if you know that I’m 5’11”, you have access to some of my personal data. I don’t regard this as highly private data, for the most point. If I walk down the street, any reasonably astute person can guesstimate my height as “around 6 foot”. The fact that they know this personal data is not a privacy violation.

Now I join a “6 foot and under” basketball league, so I’m going to allow them to measure me to make sure I fit the requirements to play in the league. Again, not a privacy violation. In order to gain access to the league, I have to provide the data, and I do so because I want to play.

Now the “6 foot and under” league sells their membership data to a data warehouse. This is useful because Rochester’s Big and Tall wants to know who to take off of their mailing list, or whatever. Here is a privacy violation. I have authorized a particular party to gather data about me in order to provide authorization/access to a service, not so that they can resell my data to anyone else.

It’s sort of a silly example, but it illustrates the point. A privacy violation occurs when personal data gathered for one explicit purpose is reused for another purpose. It doesn’t matter how “important” or “private” the data is… what is important to me may or may not be important to you. You may not care if you’re labelled in a database as “under 6 foot” or “HIV positive” or “does most shopping on Saturday” or whatever. I may care. For me to create an access policy based upon my idea of what is important is a justice violation -> I’m denying you the right to protect what you consider to be private, and enforcing only my right to protect what I consider to be private.

As long as we’re quibbling over what qualifies as “private” vs “personal” in terms of the data (as opposed to in terms of the use of the data) we’re not going to get anywhere.

All of your data is personal. It becomes private only when examined in the context of access (who needs to know the data and do I want them to know the data).

We need to examine the problem from the standpoint of access and authorization, instead of trying to define what ought to be accessed or what should have “implicit” authorization.

Knucklehead • March 7, 2006 12:15 PM

Wm,

You’re point re: enumerated powers being granted and the rest left to the states or the people is, of course, true in a purist sense. It is, however, not entirely true in a pragmatic sense.

The constitution, whether any of us believe it should be thus or not, is a “living document”. How we conduct the affairs of our government within its framework has changed, and continues to change, to meet the our needs in a changing world. Purist “originalism” is not, and has never been, possible and our legal history suggests there has never been overwhelming popular support for purist originalism.

The word “private” or “privacy” is not used within the constitution yet we clearly recognize such a “right” based upon interpretive reading of the fourth, fifth, sixth, and fourteenth amendments.

By and large the fourth amendment, Search and siezure, is the guideline for matters such as the one under discussion here. Clearly the framers and founding fathers could not possibly have anticipated the technological developments of either communications or surveillance technology (BTW, the constitution doesn’t mention “surveillance”, only “search” and “siezure” and qualifies those with the word “unreasonable”).

We cannot simply point to the constitution when discussing such matters. We have to refer to how caselaw has developed through our history. Even a cursory examination of caselaw around the matter of “unreasonable search and siezure” demonstrates that ideas and perceptions have changed. It was once thought to be purely a matter of property – if it wasn’t property it was not within the scope of the fourth amendment.

That view changed, over time, to encompass far more nebulous ideas such as “privacy” and communications. It had to. Electrons moving across wires or radio waves through the air that we don’t own can hardly be considered our “papers” or “property”.

We do not own the CDRs that describe the telephony and other network communications we have engaged in. We can’t demand that they be subjected to some purist notion of “privacy” any more than we can demand that all the various forms of communication we engage in. None of us are free to just up and build a radio transmitter of whatever type we chose and blather anything we wish out into the ether. We have developed, and we have long accepted, restrictions on communications privacy.

We do not forbid law enforcement from engaging in surveillance. A simple drive down any highway should make that clear. Law enforcement officials are perfectly free to watch you and zap you with radar and sonar and laser and whatever else they chose to use to keep an eye on you even though you are engaged in nothing the least bit nefarious. Police run surveillance stakeouts all the time.

The federal government, via the US constitution, is explicitly forbidden to engage in “unreasonable search and seizure”. What constitutes “search”, “seizure”, and “unreasonble” is subject to changing interpretation. Google up fourth amendment caselaw and read about how those notions have varied over time.

Purist originalism is meaningless. It doesn’t protect anyone from anything. The precedents of caselaw, which sometimes is not fully decided until reaching the SCOTUS and even then is subject to changing interpretations, is what matters.

Knucklehead • March 7, 2006 12:57 PM

Let’s just construct an example and perhaps we can get at why some people find the idea of this AT&T database (from the original article it seems to be a collection of more than one DB).

Some specops or secret team of some sort kick in a door in Peshwar and grab some middling a-Q or other terrorist scum. They grab his cell phone and his computer and they start figuring out who this clown is communicating with. In there they discover five US telephone numbers. So they contact AT&T and they run the numbers through the DB(s) and they get a list of all the numbers that were called by, or placed calls to, this terrorist’s cellphone. They also get a list of all the numbers those phones called or well called by. Now they start matching those numbers, as far as possible, with people, and start having a deeper look at those people and mining the CDRs to figure out if these people have a calling pattern that looks like they might be operating as a “network”.

Can anyone provide examples of why that is not a good thing? I realize this grates against some people’s notions of the nebulous idea of “personal privacy” and has some vague wiff of “spying”, but what are some examples of real, tangible harm?

Pat Cahalan • March 7, 2006 1:46 PM

@ Knucklehead

Can anyone provide examples of why that is not a good thing?

Joe and Jack are brothers. Joe left when he was 18 to go to a U.S. anniversary and has adopted his new country even to the extent of becoming a citizen. Jack stayed in . Jack became a member of Al Quaeda (unbeknownst to Joe). Joe is a devout Muslim and also a pacifist and has no ties with any terrorist organization.

Jack has called Joe on a number of occasions to talk about Grandma’s chicken pot pie.

Jack goes to Iraq to do his part in the holy war, gets nabbed by a Marine Force Recon unit and his cell phone gets analyzed.

Now Joe is on a list of potential terrorists. Moreover, Joe is only one step removed from a potential terrorist, so any organizations that Joe belongs to (such as his alma mater’s Organization for Muslim Brotherhood, Amnesty International, or the community theater company he belongs to) are only two steps removed from a terrorist.

Joe is now on the no-fly list. Anyone who is a member of any organization that Joe is a member of who has any other “red flag” anywhere in his or her profile is on the no-fly list. Joe’s buddy Jesus (who dropped out of college because he partied too much and started a groundskeeping and landscaping company) gets a visit from guys in riot suits because Jesus buys fertilizer in bulk.

Part of the problem with this is it’s all conjecture, sure. But we don’t know how people get on the no-fly list. We don’t know how people get on the terrorist watch list. We don’t know how to get off those lists if we’re mistakenly put on them.

Knucklehead • March 7, 2006 3:01 PM

Pat,

Thank you for participating.

As I mentioned above this site doesn’t seem to allow html so I can’t link. A google of “no fly list” produces some background information about the TSA lists. The EPIC site provides a pretty good level of detail. The “no fly list” was started back in 1990, seems to be a preposterously flawed program, and came under the auspices of the TSA when it was formed a few months after 9/11/01. EPIC produces a lot of documents, heavily redacted, that make it difficult to follow closely.

There is no doubt that there is a program, no doubt that there are names on the list (20,000 is the largest number I spotted) and no doubt that a lot of the names don’t belong there. What isn’t clear is how the list was populated or how one is removed from the list. Nothing I was able to spot in my cursory exploration seemed to indicate a scale of program that is enormous.

There were some 640 million passengers carried by US airlines in 2005. Every single one of those was inconvenienced to varying degrees. Some portion of those were inconvenienced due to the “no fly” and “selectee” lists. From what I can tell from a simple web search and scan of articles this seems to be a matter of dozens of people rather than hundreds or thousands.

This sounds like a program with a lot of room for improvement. I fly and there is a lot of room for improvement in the TSA programs.

But neither the seemingly ridiculous security procedures or the “no fly list” seem to represent existential threats to the republic or the basic freedoms of its citizenry. We managed, somehow, for 640 million passengers to get hauled around on US aircraft.

Joe and Jack. Jack joined a-Q, Joe doesn’t know this, Jack calls Joe for completely harmless reasons. Let’s ignore for a moment that monitoring those calls may potentially lead to some knowledge of where the heck Jack is. Let’s just leave it at Joe being unjustly paid attention to by law enforcement agents of the US government.

We may presume that any organization with “Muslim Brotherhood” in its title is subject to some level of surveillance. The Egyptian originated “Muslim Brotherhood” is, after all, one of the more notorious jihadist groups. Joe’s participation in such a group has probably been looked into regardless of phone calls from Jack.

Agents of the federal government keep an eye on Joe for a while and perhaps even have a good look at his community theater group. Ummm… so what? Has anyone prosecuted Joe for something? Is he being subjected to any more surveillance scrutiny than the brother of a known criminal would be if he were in contact with the criminal? Has Joe been hauled in and questioned about the whereabouts of his a-Q brother, Jack?

This may be very inconvenient for Joe but nothing you’ve suggested so far is anything that wouldn’t have been common actions at many times in US history. Convert Jack and Joe to people of German ancestry during WWII and Jack returned to the fatherland prior to the war and somehow manages to occassionally send a telegraph to Joe. Joe would have been placed under surveillance. If Joe were Japanese he might have been rounded up and interred in a camp.

The nation has survived this sort of thing before. It will survive it again. Just looking into the “no fly” list quickly shows that people complain, these things become, at least to an extent, known to the public. Congresscritters complain, memos get sent, FOIA filings are made and, eventually, responded to.

All that suggests that the system, imperfect as it is, actually works to make sure that widespread, or even statistically significant numbers of, trampling of the rights of Americans does not occur. In a nation of nearly 300 million people and a world where terrorists have demonstrated their desire and capacity to commit mass murder, the unjust inconveniencing of a remarkably small number of people is, perhaps, more an indication of a healthy system than an indication of a sick system. It would be nice if no American were ever subjected to anything unjust but that just isn’t reality.

Now let’s look at this from another angle. Jack’s phone number is discovered as are his ocassional phone calls to brother Joe as well as the other four numbers I posited. Using the information gleaned from mining CDRs it is discovered that Jack, or at least his phone, seems to wander around Pakistan, Afghanistan, and Algeria. US agents turn this info over to agents in those countries and ask for help figuring out who Jack is talking to.

They also find out that Jack sometimes calls a couple other people no more dangerous than his brother Joe. One of the numbers he calls, however, is a cell phone that belongs to someone who’s name matches an alias for someone who was in the US on a student VISA and more or less disappeared a year ago. The records show that the cell phone has operated in a small number of US locations and with only a small number of other cell phones in other locations. Location based info allows law enforcement to get the warrants necessary to monitor this domestic traffic, they easily track down one or two of these cell phone users and intitial surveillance suggests they might be up to no good. In the meantime Algerian authorities relay the info that they’ve got Joe under surveillance and he seems to be in touch with known terrorist cells within Algeria.

Was that potentially worth whatever inconvenience Joe experienced due to the program or is nothing ever worth any inconvenience to an innocent American?

Pat Cahalan • March 7, 2006 5:12 PM

@ Knucklehead

Was that potentially worth whatever inconvenience Joe experienced due to the
program or is nothing ever worth any inconvenience to an innocent American?

This is very difficult to judge, because we don’t know

  • (a) How many terrorists are caught using these methods
  • (b) How many of those terrorists would have been caught anyway
  • (c) What would be the results of having these terrorists “still on the street”.
  • (d) How many innocent people are affected by these methods
  • (e) The degree of the imposition put upon those innocents

Let me turn your question on its head, and let’s make up some numbers. Say there’s 20,000 innocent people on the no-fly list. Maybe 1 is a doctor, who missed a surgery because he missed a flight and someone died. Maybe 1 is a kidney donor, who missed a flight and someone died. Let’s say there’s 500 people wrongly deported, or had their visas revoked. Let’s say we’ve refused visas to hundreds of scientists and cryptologists because of their country of origin, preventing them from sharing their expertise (from which we would benefit). Maybe there’s 1 or 2 people who have been “relocated” to other country’s intelligence services for “further investigation”.

Maybe, for all that, we’ve caught one terrorist who was never going to operate in the U.S. during the course of his career, or maybe we would have caught him anyway, or maybe he’s stupid and would have blown himself up trying to mix up an IED in his apartment.

Is that worth it? How do we know? What metrics are we using to measure the costs and benefits of these sorts of programs?

Joe • March 8, 2006 12:32 PM

I find it odd that most comment-ers think AT&T is only keeping this information around because of security or regulatory needs.

It costs a lot of money to build and run a large telecommunications system. AT&T probably built this database primarily to understand how their network was used, and to predict future usage trends. AT&T had a long history of doing this type of thing: remember that queueing theory was developed at Bell Labs to help model usage and optimize resource allocation.

Nearly1984 • March 15, 2006 5:24 PM

SBC is a heavy user of the Teradata massively parallel database platform – Teradata is an ideal platform for the retention and analysis of data of this nature.

Numerous “national security” organizations employ this technology to analyze huge stores of data (100s of TB OF ACTUAL DATA). Sure it’s expensive but don’t think for a minute that it’s infeasable.

If the government can do it then so can corporates…

BILL • May 11, 2006 10:59 PM

Looks like we are all in the same boat as the crime syndicates were. The only way to keep this private is thru conversations in the garden and letter mail.

good time man • February 13, 2007 12:45 AM

For now I’m working for this low life company. I did think ( years ago ) that AT&T was a company that had integrity and cared about customers and especially employees.
Well, working there in a business office with almost no limits to customer records, I now know that they don’t care about anything except making a buck.
I’ve seen them completely lie to customers and I’ve been instructed to lie to customers. It’s bad when you don’t want to screw a customer and tell them a lie.. Like ” I’m sorry but this is the only type of service that will work and is compatible with your equipment”, this kind of thing is horrible.
I do know that AT&T would keep and harbor any type of records at any of the governments request. They have marketing groups that cater strictly to government so since they make the upper management wallets fat, they will do anything and everything asked.
We get daily emails of what to tell customers if they ask about this or that. You can’t just tell them anything, it is a script. It tells you exactly what to say if asked certain questions, you can’t just tell the truth anymore, not working for AT&T.
I think that’s why they will end up breaking this giant company who has morphed into a superpower once again. Not because of competiton but you just can’t sit around and let evil grow into something so large. It’s easier to deal with in smaller pieces.
The union does try to represent the employees interests, however, they don’t really have a leg to stand on. They break contract and then have you sign things to protect themselves.
Of course, your not under duress or coersion.. That’s what every legal document says. But we all know what type of big brother company does this to employees. And the union just says ” I’m sorry, we did the best we could, it’s wrong that they do this”. What a weak and pothetic union. I bet by 2010 there will no longer be a union and they will take away your weekend, along with your soul!!!
Databases- they have them. That’s one thing they are best at. They have about 100 databases that only manage other databases. This company really is evil.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.