Automatic Profiling Is Useless

No surprise:

Automated passenger profiling is rubbish, the Home Office has conceded in an amusing—and we presume inadvertent—blurt. “Attempts at automated profiling have been used in trial operations [at UK ports of entry] and has proved [sic] that the systems and technology available are of limited use,” says home secretary Jacqui Smith in her response to Lord Carlile’s latest terror legislation review.

The U.S. wants to do it anyway:

The Justice Department is considering letting the FBI investigate Americans without any evidence of wrongdoing, relying instead on a terrorist profile that could single out Muslims, Arabs or other racial or ethnic groups.

I’ve written about profiling before.

Posted on July 7, 2008 at 1:37 PM28 Comments

Comments

dragonfrog July 7, 2008 2:07 PM

Seems reasonable – if you single out Arabs because they look Arabic, it’s racial harassment. But if you single them out because your software tells you to, you have applied a sufficient layer of magic that it’s no longer racism.

(even though your software simply does a check for Arabic background)

Secret Agent Man July 7, 2008 2:18 PM

Is there a way for the FBI to respect privacy (and the 4th amendment) and still surreptitiously go fishing for, and collect intelligence on, percolating plots?

FP July 7, 2008 2:28 PM

No problem, you just do the profiling in secret. The NSA then just tips off the FBI about everything it might consider suspect, which for the FBI is probable cause to investigate.

Devil's Advocate July 7, 2008 2:31 PM

The reason that passenger profiling is crap is because of a lack of data, so we need more of it to base the profile on, and because there is too much privacy red tape, so we must have less of it.

mu July 7, 2008 2:57 PM

dragonfrog said:
“Seems reasonable – if you single out Arabs because they look Arabic, it’s racial harassment. But if you single them out because your software tells you to, you have applied a sufficient layer of magic that it’s no longer racism.”

Does anyone remember something like this in an HR system a few years ago? I think there was an expert system for filtering job applicants; human applicant screeners were used to build the training set but the humans were biased (consciously or not) against brown people.

I think I came across this in comp.risks but the closest things I can find in the archives are:

http://catless.ncl.ac.uk/Risks/16.19.html#subj3.1
http://catless.ncl.ac.uk/Risks/16.20.html#subj8.1

George Smiley July 7, 2008 3:00 PM

It’s not useless. It’s just that the function is not security. The function is to exert power over the Other – to let ’em know who’s in charge. Profiling is a notably efficient way to accomplish this function.

Secret Agent Man July 7, 2008 3:16 PM

@FP – are you saying the way to keep the FBI following the rules is to have the NSA break the rules for them?

My question should have been about the “US Gov’t” instead of “FBI”.

Jason July 7, 2008 3:19 PM

@Devil’s Advocate

The more data you collect, the more difficult it becomes to manage and the more time consuming it becomes for useful queries to return meaningful responses.

It isn’t impossible, but it is a difficult problem and success within the current political framework is highly unlikely.

Also, the “more data” philosophy assumes that human beings are capable of developing a correlation engine that can distinguish between coincidence and direct intent.

This is the same government that gave us the “No Fly List” (see Schneier’s own essay from 2004 – http://www.schneier.com/essay-052.html)

Nomen Publicus July 7, 2008 4:05 PM

Remind me, what was the nationality and physical characteristics of the man who left a car bomb in front of the Federal building in Oklahoma City?

Surely the point is to detect, and prevent all such acts no matter who is involved.

Anything that ignores a large percentage of the population is a foolish policy that will just result in more outrages.

Anonymous July 7, 2008 4:19 PM

@Nomen

Anything that ignores a large
percentage of the population is a
foolish policy

Don’t worry, they’re also tracking people who rent trucks, people who buy fertilizer, and people who speak Latin.

SteveJ July 7, 2008 4:27 PM

@Nomen: “Anything that ignores a large percentage of the population is a foolish policy”

I realise you didn’t mean it this way, but anti-terrorist police should be ignoring a large percentage of the population a large proportion of the time. Including a large proportion of Arabs. The alternative is to conduct surveillance of a large proportion of the population…

Carlo Graziani July 7, 2008 5:05 PM

Perhaps the FBI can hook up their automatic profiler to that automatic hand-held polygraph machine, creating a chained techno-magical application that can sniff out terrorists and bust them in one mystical button-press.

mission-is-critical dept. July 7, 2008 5:08 PM

There’s AI that is fairly effective for this sort of thing.
“In the fall of 1983, Special Agents
from the FBI’s Behavioral Science Unit
constructed a criminal personality
profile describing an individual who
could have been responsible for a series
of fires at religious homes and houses
of worship that summer in a posh New
England community. The profile was
prepared at the request of the com­
munity’s police department, which later
discovered that the FBI’s profile not only
accurately described the suspect in
detail but also pinpointed his residence,
based upon a series of, intricate
computer calculations using artificial
intelligence technology. The suspect
later confessed to the crimes.”
DAVID J. ICOVE, Ph.D., P.E.
Senior Systems Analyst
Behavioral Science Investigative Support Unit
National Center for the Analysis of Violent Crime

Do enough calculations and you are likely to gain an advantage. Automate it and the data creates a path of logic.

dragonfrog July 7, 2008 5:43 PM

@Carlo Graziani

They just need to tie in an auto-convictor, and the whole magico-judicio-terror-fighting process can reach new levels of efficiency.

fbi--drone-in-my-yard dept. July 7, 2008 5:57 PM

magico-judicio-terror-fighting
“Sittin out back enjoying this night. See what looks like a large insect flying round about 300 yards off my back porch. Now, I am watching it intently. It is watching me intently. It gets closer and has a large rotating eye underneath. The whole thing is the size of a surboard. Now it is about 30 yards off and about 50 feets high. A slight mechanical hum. It turns on a light. Now, this is no ordinary light. It was the color of Pepto Bismal, and it focused very intensely upon me. I yell out “What you want? Stop puttin the light on my eyes!” It did not respond at all. Needless to say I am extrememly worried and angry at this point.

Next thing I know, i feel like someone just let me know “everythings ok. We’re not here for you.” It felt like someone just took a ton of bricks off my shoulder. The miniature aircraft/drone is gone. My clock says its almost 11pm, and I know I saw this thing starting at sundown! The federal government has been in my backyard and they came in with some kind of drone to probe me and my property!”
http://humboldt.craigslist.org/rnr/730554482.html

Davi Ottenheimer July 7, 2008 6:34 PM

Ah, let’s not forget the news from ICE (Immigration and Customs Enforcement).

Here’s one of their latest press releases from ICE, where they arrested eight people on criminal charges:

http://www.ice.gov/pi/news/newsreleases/articles/080208losangeles.htm

In short, more than 100 federal agents entered and sealed the perimeter of a business with 130 employees.

Everyone was detained and forced to prove their identity/right to work.

Here is a typical critique of ICE, related to the same incident:

http://www.hispanicbusiness.com/forum/topic.asp?TOPIC_ID=20884

“…I.C.E. agents neglected to honor these victim’s 4th Amendment rights. For instance, one lady requested not to speak to the agents unless she had an attorney present. They told her she had no right to an attorney, which goes against our 4th Amendment Constitutional Rights. Another was taken for days, while her daughter was left to fend for herself.

Guest speakers also included the brother of Pedro Guzman, a U.S. citizen who was deported to Mexico by I.C.E. He elaborated on the fact that a judge had ordered that Pedro Guzman (who has memory problems that requires family assistance) be turned over to his family and STILL I.C.E. deported him to Mexico, clearly ignoring the judge’s order. If the ACLU hadn’t become involved, this U.S. citizen who was born and raised in California might still be wandering in Mexico.

They addressed the cruel tactics used by I.C.E. in dealing with the detained, waking them up in the middle of the night for lengthy interrogation practices and causing them sleep deprivation; holding them on buses for five hours in the hot summer without even allowing them to use a bathroom; leaving kids homeless for months before they were found and turned over to Social Services.”

As much as I would like to say profiling is terrible and useless, there is obviously some need for a sketch or a profile for law enforcement to narrow their search.

Perhaps fishing is a good analogy. While a spear or hook is the best approach to catch a single fish, it is not usually practical if you face vast territory with small resources. On the other hand, using a giant dragnet is incredibly destructive to other sea-life and potentially a greater harm than letting the fish you want go free. There is surely a balance somewhere in the middle, but so far it looks like the US Administration has favored the latter approach rather than finding a balance.

Papers July 7, 2008 7:20 PM

@Davi

Here is a typical critique
Are you seriously citing an uncorroborated comment by an unidentifiable person in a self-moderated blog as an authoritative account?

she had no right to an attorney, which
goes against our 4th Amendment
Constitutional Rights.
Fourth Amendment isn’t about attorneys, it’s about warrants (which ICE apparently had).

I have no doubt that ICE can be mean/unfair/stupid/Orwellian, but please use more credible, verifiable references.

Anonymous July 7, 2008 7:43 PM

My favorite quote:
“Currently, FBI agents need specific reasons, such as evidence or allegations that a law probably has been violated, to investigate U.S. citizens and legal residents”

ElectronicMessiah July 8, 2008 12:17 AM

I’m current reading Bruce’s book, “Beyond Fear”, and I recall a idea from today’s reading that basically said something like “the more simple the security, the better it is” (less to go wrong). Instead of capturing tons of data that can never be analyzed and relying so much on technology, perhaps the feds should just train their employees/TSA to watch people. I’ve had enough psychology to know the signs of people under stress, lying, trying to create chaos…they can’t usually hide it very well.

There are some warning signs that are easy to look for (one way tickets, no luggage, groups of people communicating quietly but keeping their distance from each other). I sat in an airport right after 9/11 and watched people while waiting for my flight. I spotted the Fed immediately as he sat and did the same. Remember, no amount of digital data can ever replace boots on the ground and sharp eyes/ears.

Anonymous July 8, 2008 1:25 AM

@Devil’s Advocate
The reason that passenger profiling is crap is because of a lack of data, so we need more of it to base the profile on, and because there is too much privacy red tape, so we must have less of it.

At best this would be useless. Terrorists are so rare that any kind of “profiling” will generate mostly false positives. It gets proportionally worst if the intention is only to catch a subset of terrorists.

MathFox July 8, 2008 5:42 AM

@Papers:
There are so many, very believable reports of abuse of power by US government officers that there must be rotten apples in the basket. There even is a small stream of documented cases where officers perjured or performed other crimes. Correcting for the difficulty of getting officers prosecuted I would say we only hear of a fraction of the abuse.

Papers July 8, 2008 11:10 AM

There are so many, very believable reports of abuse of power
… I would say we only hear of a fraction of the abuse.

I’m sure you’re right – much underhanded activity we never hear about. But we also hear twisted and distorted stories about legitimate activities.

The automatic propagation of rumors without any attempt to verify them really does truth a disservice.

Doug Coulter July 8, 2008 12:24 PM

As someone on slashdot just said — I don’t know anyone who is afraid, where are all these frightened people who are asking the government to do this stuff? They concluded that the government is doing it TO us, rather than for us.

I agree. Just about none of this is stuff a government does to protect citizens from an EXTERNAL threat. You know, the type they’re telling us to fear.

It is very much the stuff a government does when it is worried about an internal revolt, however, and history seems to be repeating on this. The examples are too numerous to single one out right now.

All this monitoring is to try to catch a possible uprising before it gains enough human mass to be credible and begin to snowball.

You could say it’s cheaper and less damaging than waiting for a real fight, in some twisted logic — it will save lives and property in the short run. And oh, the incumbents won’t have to lose the power they crave so dearly.

Better for everyone to simply agree to live under a Fascism no one (sane) wants? To be able to “vote” for candidates no one wants? In a system where the fox counts the chicken’s votes and selects who can even be voted for?

Bah. We’re cursed with interesting times.
I’d much rather be playing in my lab, and my garden, doing something useful and productive, so, for now, I will.

I really fear for this country, and some others. Used to be you could just leave and go someplace saner. Doesn’t seem those places exist anymore.

Davi Ottenheimer July 8, 2008 2:13 PM

“Are you seriously citing an uncorroborated comment by an unidentifiable person in a self-moderated blog as an authoritative account?”

Don’t know what you would consider authoritative so I leave it to you to do the research.

I have read numerous cases and articles and listened to first-person accounts on the radio that all suggest the same thing — if someone carrying a badge knocks on your door at night you need to exercise your constitutional rights or they will enter and detain everyone until “authoritative” residency/work documentation is proven. It is likely that anyone unable to show papers, including those requiring medical assistance or with small children, will be forcibly deported within 24 hours.

Even ICE does not disagree with the accounts, they just report them as success stories and statistics. Note the “humanitarian services” numbers that they now include in their press releases. This is specifically meant to offset the growing criticism, which I guess you could say is a good thing (if it is more than a marketing tactic).

Frances July 8, 2008 8:20 PM

@Anonymous at 7:42 a.m.

“Attempts at automated profiling have been used in trial operations [at UK ports of entry] and has proved [sic]…”

I think that [sic] is in the wrong place. “Attempts” is plural so it should be “have proved”.

You are right that it sounds better; it’s because it is better.

yonodeler July 11, 2008 1:58 AM

That comments regarding placement of “[sic]” not leave unchallenged red marks (however trivial) against the Schneier post above, I observe that the insertion of “[sic]” between an auxiliary (helping) verb and the main verb that follows it would be ungainly and possibly confusing.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.