Yet another massive U.S. government database -- OneDOJ:

The Justice Department is building a massive database that allows state and local police officers around the country to search millions of case files from the FBI, Drug Enforcement Administration and other federal law enforcement agencies, according to Justice officials.

The system, known as "OneDOJ," already holds approximately 1 million case records and is projected to triple in size over the next three years, Justice officials said. The files include investigative reports, criminal-history information, details of offenses, and the names, addresses and other information of criminal suspects or targets, officials said.

The database is billed by its supporters as a much-needed step toward better information-sharing with local law enforcement agencies, which have long complained about a lack of cooperation from the federal government.

But civil-liberties and privacy advocates say the scale and contents of such a database raise immediate privacy and civil rights concerns, in part because tens of thousands of local police officers could gain access to personal details about people who have not been arrested or charged with crimes.

The little-noticed program has been coming together over the past year and a half. It already is in use in pilot projects with local police in Seattle, San Diego and a handful of other areas, officials said. About 150 separate police agencies have access, officials said.

But in a memorandum sent last week to the FBI, U.S. attorneys and other senior Justice officials, Deputy Attorney General Paul J. McNulty announced that the program will be expanded immediately to 15 additional regions and that federal authorities will "accelerate . . . efforts to share information from both open and closed cases."

Eventually, the department hopes, the database will be a central mechanism for sharing federal law enforcement information with local and state investigators, who now run checks individually, and often manually, with Justice's five main law enforcement agencies: the FBI, the DEA, the U.S. Marshals Service, the Bureau of Prisons and the Bureau of Alcohol, Tobacco, Firearms and Explosives.

Within three years, officials said, about 750 law enforcement agencies nationwide will have access.

Computerizing this stuff is a good idea, but any new systems need privacy safeguards built-in. We need to ensure that:

  • Inaccurate data can be corrected.

  • Data is deleted when it is no longer needed, especially investigative data on people who have turned out to be innocent.

  • Protections are in place to prevent abuse of the data, both by people in their official capacity and people acting unofficially or fraudulently.

ln our rush to computerize these records, we're ignoring these safeguards and building systems that will make us all less secure.

Posted on January 2, 2007 at 11:55 AM • 17 Comments


Amos NewcombeJanuary 2, 2007 1:07 PM

OneDOJ to rule them all, OneDOJ to find them,
OneDOJ to bring them all and in the darkness bind them
In the Land of DOJ where the files lie.

Jemaleddin ColeJanuary 2, 2007 1:24 PM

I think, from the DOJ's perspective, the entire point of OneDOJ is to preserve information about innocent people. Or rather, that with OneDOJ, the assumption is that nobody is ever really innocent - just waiting to be found guilty.

Really warms the cockles, eh?

BenJanuary 2, 2007 1:49 PM

"Data is deleted when it is no longer needed, especially investigative data on people who have turned out to be innocent." is never going to happen, as Jemaleddin Cole says. No one is innocent in a system like that, everyone is either a suspect or a criminal.

What would the agencies get out of removing information from it? Who decides when someone is innocent, since the presumption of innocence is gone?

Someone needs to bribe, borrow, or blackmail access to one of the machines, then look up every congressman and all of their spouses and children and parents and crazy uncles. And then send all that information to The Smoking Gun.

BobJanuary 2, 2007 1:54 PM

If it makes you feel any better, this won't be implemented any time soon. I work for the largest industry vendor of crime information systems, and OneDOJ has a LONG way to go before any large agencies will completely adopt it.

Davi OttenheimerJanuary 2, 2007 4:19 PM

@ Bob

Can you share more details? The DOJ officially has painted a slightly different picture in their "E-Government Status Report - FY-2006":

"In 2006, OneDOJ’s internal user community increased to nearly 400 investigators and analysts spanning all investigative components. Two additional partnerships were established in 2006 with key regional sharing initiatives including: The Automated Regional Justice Information System (ARJIS) of the San Diego Region; and Southeast Law Enforcement Information Exchange (Southeast LInX), which includes the Naval Criminal Investigative Service (NCIS) and law enforcement agencies across Northeast Florida and Southeast Georgia. DOJ’s case information is now being shared with thousands of authorized law enforcement users. In preparation for a more aggressive partnership rollout plan for 2007, DOJ has developed a streamlined process to establish a substantial number of additional regional partnerships with federal, state and local sharing initiatives. "

More aggressive rollout and streamlined? Sounds rushed and potentially unsafe to me (to echo Bruce's point above).

Democracy is FragileJanuary 2, 2007 5:47 PM

Bruce wrote:
>...any new systems need privacy safeguards built-in. We need to ensure that:
[some good stuff deleted]
>Protections are in place to prevent abuse of the data, both by people in their official capacity and people acting unofficially or fraudulently.

I totally agree with these ideas. These safeguards were woefully missing from the TSA no-fly lists, etc.

But here's what haunts me: Is it really possible to prevent abuse of data to the degree necessary to ensure liberty?

We're seeing an unprecedented proliferation of government tracking of its citizens.

At a time when you have an Executive branch thumbing its nose at Congress over warrantless wiretaps, how can we possibly trust them (and *all* *future* *administrations*) with the kind of databases that are so ripe for politically-motivated misuse?

Nixon had his enemies list, and Hoover's FBI had files on everyone from John Lennon to Martin Luther King, Jr. These are facts that should give us pause. They aren't even close to the worst-case scenario.

IMO, those who say "why worry" are just ignoring both the past and the inevitable future abuses.

So yes, we should put reasonable safeguards in place for any government database. But we should also think pretty hard in each case about whether to have those databases in the first place.

RonKJanuary 3, 2007 12:26 AM

@ Amos Newcombe
> In the Land of DOJ where the files lie.

Wow, that one gets enshrined in the repository of classic double entendres....

Way to go!

C GomezJanuary 3, 2007 7:54 AM

Don't you think this is always the competing self-interest at work? Law enforcement has a priority to catch bad guys... and very little else. This doesn't make them bad people. It's just their interest.

I don't think any rush to build a computerized database has anything to do with lack of privacy safeguards. I think even if it were built "slowly" you'd have to convince law enforcement that it's not enough to let them be the sole "benevolent stewards" of the data.

Democracy is FragileJanuary 3, 2007 2:15 PM

@C Gomez,

I'm not quite sure what you're advocating here. Bruce isn't accusing Law Enforcement of bad will. He's addressing the social risks of letting them operate without proper checks and balances.

The police do their jobs while obeying the Fourth Amendment, Miranda rights, restraint in use of lethal force, etc. They are professionals after all.

Granted, law enforcement agencies have had these restraints imposed on them by Congress and the Supreme Court.

So if what you're saying is that it's not in their nature to voluntarily put checks on their own power to get the bad guys, then we agree.

JemaleddinJanuary 3, 2007 4:03 PM

The question isn't whether the folks in the FBI/DOJ/LAPD are "good guys" or "bad guys." There was a case a few years ago where two FBI agents in New England used *the existence* of open investigations to blackmail several companies. Not wrong-doing. Not malfeasance. Just the fact that FBI was checking around made these guys money. How many other people will be willing to pay to keep their secrets? And how many "good guys" will be late on their child support payments and start thinking about making a little extra money off of the "bad guys"?

The other question we should be asking ourselves is: what good will this system do? It's not like much of the info will be admissible in court. So the cops pulled you over for speeding and let you off with a warning in Phoenix last year - what does that have to do with this murder case in New York? The problem with systems like this isn't that there isn't enough data - it's weeding through it to get something useful. And frankly, we aren't good at that yet. I think somebody's been watching too much CSI or 24.

Matt KJanuary 3, 2007 4:58 PM

Here is something that I don't understand about this.

You can't expect the FBI to *not* use a central database for tracking their cases. It only makes sense that the FBI, in their efforts to improve their computer systems, will use a database to make sure all of their agents can find links between cases, etc. That internal database will obviously contain names of suspects who will eventually be ruled out.

The next logical step in improving DOJ efficiency would be to make that internal FBI database accessible by all of DOJ (Marshals, DEA, etc).

The next logical step would be to give state and local agencies access to the same data.

My point with all of this is that it seems like a logical progression of data sharing that is drawing fire for sharing data. The comments here and those in the media suggesting that OneDOJ shouldn't exist at all don't make sense to me. Are you saying the FBI shouldn't have databases to track cases or are you just saying that they shouldn't share that database with the rest of DOJ/state/local? If you're saying that they just shouldn't share, that does nothing to prevent abuse by the FBI, it only prevents abuse from DOJ/state/local. You need the safeguards whether the database is shared or not. Limiting sharing of info is an artificial roadblack to abuse that is still possible.

Correcting inaccurate information is always a good thing. Bad data is bad for citizens and bad for the police. However, how do you determine "innocence" and then delete data? In the context of a single case, a person may appear "innocent", but in the context of several others, that person may suddenly become very interesting.

If you put too many rules on and remove data from a database of this type, you automatically reduce it's usefulness. I'd like to know more about what safeguards could be put in place that would protect citizens while keeping the database useful.

FuzzyJanuary 3, 2007 7:22 PM

I am a bit confused.

If the FBI is unable to get their Virtual Case File system to work after $100 million spent (see ) then how does the DoJ expect to join the databases from multiple agencies into a single working interoperable massively distributed database with proper access controls?

Is this going to be any more effective than the now defunct MATRIX program and will there be any better controls over accuracy and access to private records?

Democracy is FragileJanuary 3, 2007 7:27 PM

Matt K wrote: "If you put too many rules on and remove data from a database of this type, you automatically reduce it's usefulness."

As you said, bad data isn't good for anybody. The TSA no-fly list showed what a nightmare it can be when there is no recourse for the innocent. And from press reports, some of the "mistakes" looked like politcally-motivated harassment. I'm sure it all seemed logical at the time, but it proved to be deeply flawed from the start.

If I recall correctly, they eventually put procedures in place for no-fly to establish innocence and correct the data. So that is possible too for some kinds of databases. (Having so much bad data to begin with put the cleanup burden on the innocent out of bad design or just pure disregard for liberty. This should not be accepted ever again.)

Matt K wrote: "I'd like to know more about what safeguards could be put in place that would protect citizens while keeping the database useful."

One safeguard is to make sure that the people who access the database do so for valid reasons. See, for example, the DHS Privacy Office report on MATRIX (it's only 9 pages, link below). Many of the recommendations apply generally. Requiring that users enter a case number when accessing the system allows for auditing. Independent auditing of usage could help maintain legitimacy without reducing usefulness.

Another safeguard is avoiding mission creep. You point out that there are "logical steps" of progress. Given proper safeguards, many of these are probably reasonable. But from the point of view of law enforcement, there is no end to this progression. It would be perfectly _logical_ to have an RFID chip embedded in each human at birth, and to track every single person 24/7.

I know that sounds like a unabomber manifesto, but the reality we face is that it's becoming technically possible. Where would you draw the line across that logical progression? More importantly, why would you even draw the line at all?

Matt KJanuary 3, 2007 7:59 PM

"Democracy is Fragile", I agree with most of your response. I certainly won't argue with having mechanisms in place to remove bad information and the MATRIX recommendations regarding detailed auditing are always a good idea for any computer system/database.

You lose me with your last two paragraphs though. We all know that the sci-fi books of yore are now possible (chips implanted, massive government control, etc), but in this case no one is proposing such drastic measures. We talking about a database system to track and share information, something businesses have been doing for years now because it makes them more efficient.

You talk about where you draw the line. Some of the postings here and some of the articles I have read on this seem to be suggesting that, out of fear, we should draw an artificial line and stop the DOJ from doing something that makes business sense for getting the job done. I agree that we draw the line when they want to put a chip in my head, but no one is suggesting that.

I think that sometimes privacy advocates do themselves a disservice by using too many "the sky could fall" scenarios. Give me an example of realistic mission creep that would concern you, not the "they're going to put chips in all our babies."

Democracy is FragileJanuary 4, 2007 12:52 AM

Matt K said: "I certainly won't argue with having mechanisms in place to remove bad information and the MATRIX recommendations regarding detailed auditing are always a good idea for any computer system/database."

Yet we see time and again that *neither* of those safeguards are put in place. I don't know about you, but that gives me pause. It tells me they aren't taking this seriously.

Not to be combative, but describing the burgeoning proliferation of government files on its citizens as merely "something businesses have been doing for years now" glosses over a critical difference: DOJ, TSA, NSA, etc. are organs of state power. As such, they can restrict your civil liberties in ways that businesses cannot.

At the risk of repeating myself, the lesson of TSA no-fly seems clear: whether by malice, malfeasance, or honest mistake, such a program can and will limit the liberty of innocent persons. That's why caution is warranted.

But the business-as-usual notion also belies the fact that we truly are entering new territory, both in business and government. Three weeks ago 800,000 people were informed by UCLA that their identies may have been stolen. That never happened when I was growing up. Don't kid yourself: this is a new era, and it's still evolving quickly.

We live in a time of Guantanamo, extraordinary renditions, torture, and secret CIA prisons. CALEA, the Patriot Act, RealID, and the suspension of habeus corpus even for US citizens (thanks, McCain!). A time when the executive branch seeks to be secretive and unaccountable. I hear you on the "chicken little" thing, but repeatedly assuring oneself that "it can't happen here" is not a diligent response to this new era either.

I don't honestly fear being swept off to Syria myself and tortured like Khalid el-Masri was because the US confused his name with a similar one on a terrorist watch list. But heads need to roll over something like that. Instead we shrug and say "after 9/11, the sky could fall if we don't do this".

Re: the avoidance of "mission creep", that is actually another recommendation of the MATRIX report. But I'll give you an example that already happened: the NSA warrentless wiretap program. That is mission creep on fire.

GregJanuary 9, 2007 7:37 PM

Watch this one, people. Be cautious. I agree that "investigations" by various agencies will pollute the database and accuracies - but your name will be there - none the less. Prepare. The inevitible conflict between freedom loving people and over-reaching (some say tyrannical) government will come sooner that we all think.

withheld by requestFebruary 1, 2007 6:27 PM

All your fears are real. A couple of weeks ago I applied to get a foreign woman's address from an on-line agency. A background check was required. No problem I have never been arrested in any jurisdiction in the world. Their "report" identified me as a sex offender in eleven states and as currently serving prison time in four different states. Where do you suppose they got their information? [true story!]

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc..