Bruce Schneier

 
 

Schneier on Security

A blog covering security and security technology.

« The Difficulty of Profiling Terrorists | Main | Vista Activation Cracked by Brute Force »

March 14, 2007

Find Out if You're on the "No Fly List"

I'm not. Are you?

Soundex works, generally, by removing vowels from names and then assigning numerical values to the remaining consonants.

This has been the basis for the Computer Assisted Passenger Pre-Screening System (CAPPS) and it is horrendously inadequate and matches far too many names. To see just how poorly Soundex performs, visit nofly.s3.com and type in your name to assess your chances of being on the No Fly or Watch List. This is the only known publicly available site for checking your name against potential terrorist identities and databases. It was developed by S3 Matching Technologies of Austin, Texas. The company's database technicians merged the best known data on terrorists with the Soundex system to create the site.

Posted on March 14, 2007 at 7:51 AM59 CommentsView Blog Reactions

To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.

Comments

I am

Posted by: Anonymous Coward at March 14, 2007 8:29 AM


I am, too.

Posted by: grrlwonder at March 14, 2007 8:37 AM


One little problem with that list: one of bin laden's aliases doesn't even show up.

Do this:
goto http://www.fbi.gov/wanted/terrorists/terbinladen.htm

Notice the alias Mujahid Shaykh

Run that one through the "watch list"

It won't come up. Anyone see a problem here or is it just me?

Posted by: Geoff at March 14, 2007 8:38 AM


Two matches highlighted in red.

But this appears to be just a stunt by s3 to promote their own data mining software.

Posted by: Wim Coenen at March 14, 2007 8:44 AM


So to Soundex "George Bush" is a match to "Greg Bisheau"?

Wonderful work there, but I wouldn't trust them with a guest list to a kid's birthday party.

Posted by: Roy at March 14, 2007 8:44 AM


And even funnier is that John Smith is flagged... the most common name in the US or UK. :-)

Posted by: Ledow at March 14, 2007 8:47 AM


I think what you all need to remember is that this doesn't flag you as a terrorist, it just flags names that could possibly from a country that could imply you need to be screened.

Posted by: Tobias at March 14, 2007 8:47 AM


And all of you that just ran your name... you are now on the list.

Posted by: No Fly List at March 14, 2007 8:49 AM


2 for 2

http://www.fbi.gov/wanted/terrorists/teralzawahiri.htm

Abdel Muaz
Abu Muhammad
Abu Fatima
Abu al-Mu'iz

Posted by: Geoff at March 14, 2007 8:53 AM


After a misleading headline, their description states this is a "demonstration" of how the No Fly list works. The fine print even states they obtained the names used in this list from published sources (not "the" No Fly List).

They are trying to sell an improved Soundex-like algorithm, of which there are many.

Posted by: Roenigk at March 14, 2007 8:55 AM


I believe Bruce fell for a hoax.

Posted by: Johns at March 14, 2007 8:56 AM


Soundex? Removing vowels and assigning number values? You mean gematria?

Posted by: Joshua at March 14, 2007 9:09 AM


I am

Posted by: Anonymous at March 14, 2007 9:14 AM


@Geoff:
Why should UBL be on it? The FBI doesn't want him for 9/11... :->

Posted by: Martin Schröder at March 14, 2007 9:15 AM


I'm trying to find this fine print that others are mentioning, but I haven't found it yet. What I have found is:

"The results generated in this demonstration are a product of a compilation of the best available data regarding suspected and known terrorists"
[Ed: I'd suspect that fbi.gov's info "should" be in there]
...
"Publicly available terrorist names from various reliable government and non-governmental sources were merged to create a comprehensive list."
...
"Like the federal government's, this list of terrorists, integrated into the Soundex software believed to be used by the TSA, is constantly updated"


From what I gather from the above it means the db is live, but they're querying it with the "Soundex" method to prove what a joke it is in order to play up their own offering (TeraMatch). However, the db (from what I'm reading in those quotes) is indeed a live db.

Posted by: Geoff at March 14, 2007 9:21 AM


Traditional Moslem names, as in the kind that someone who's angry with the West would maintain, aren't constructed in the typical western firstname-lastname pattern. Someone born in Persia, Magrib, Arabia or Hindustan (they don't like our country names either) may have seven names, several of which would commonly be used, either alone or in combination, to identify them. Compound this with the fact that there is no single recognized method for transliterating Arabic into English. Then consider that sneaky people sometimes have aliases.

By the time you enter a first name and last name of a Moslem terrorist into this system, you're as likely to get a match as if you entered random characters.

I wish John was right about this being a hoax, but I fear he's not. We're 18 days early for April Fools' Day.

Posted by: Stephan Samuel at March 14, 2007 9:32 AM


I don't understand the list. Why all the names not shown in red? Richard Cheney. Richard is red, cheney is not. Why bother to show cheney then.

Interestingly my first and last names are red. My first, stephen, is covered by multiple red lines for variations. My middle Neal is red. My last is red. I guess that doesn't bode well for me should I ever fly. Haven't yet at 42 though so....

Well, off to Google my name and variations of terror and the like to see if I can find someone close in name.

Posted by: Neal at March 14, 2007 9:34 AM


When I wrote the Lingua::EN::MatchNames Perl module, I used Soundex as a last desperate attempt to match names, with an incredibly low confidence level. I found the Metaphone algorithm to be far, far more accurate.

Soundex sucks.

Posted by: Brianary at March 14, 2007 9:34 AM


@Wim Coenen:
@Roenigk:

"...the federal government is using an algorithmic software product known as Soundex to search databases for potential terrorists."

"TeraMatch® matched with 96% accuracy compared to Soundex's 15%. Importantly, TeraMatch® only matched 4% false positives-innocent people incorrectly identified-to Soundex's 85%."

Are they trying to sell their software? Yes, but they have a very good point about the Feds using perhaps the stupidest approach available to implement their matching algorithm.

Posted by: Brianary at March 14, 2007 9:47 AM


Nice... especially since people tying their names in also expose their IP address to the name they are searching.

Do the math.

Israel Torres

Posted by: Israel Torres at March 14, 2007 10:27 AM


Let's say your surname is Laden, and your first name is Osama Bin. That's right, you're not on the list!
If you have a non-European name, moving your surname around shouldn't be that hard.

Posted by: Proops at March 14, 2007 10:37 AM


I call shenanigans! It looks like that system mathces first names to 'suspected' first names, and last names to 'suspected' last names, but doesn't compare the first/last pair to a list of 'suspected' first/last pairs. So "John Anything" compares to, wow, quite a few... So does "anon ymous".

Posted by: wiredog at March 14, 2007 10:42 AM


@Johns,

> I believe Bruce fell for a hoax.

The preeminent security guru of our time? One of the leading lights on the internet? SURELY NOT.

Posted by: Anonymous at March 14, 2007 11:09 AM


The list is obviously broken. Osama bin Laden is on it, but so are Bob Dole, George Bush, and Al Gore.

It's a fun demonstration of Soundex, but that's all.

Posted by: Michael Ash at March 14, 2007 11:19 AM


Only the most common names appear to be on the list.
So Maria, Mary etc are in nofly first names. Too bad if the last name also happens to be Smith or Lopez if the real database would be that plain ...

Posted by: a. at March 14, 2007 11:28 AM


even "anonymous coward" is on the list :D

so half of /. posters wont ever be flying again....

Posted by: anonymous coward at March 14, 2007 11:39 AM


I'm suspicious of this story; it looks like PR for a company that's trying to get the feds to buy their software. They claim, for example, that my name is a match, and I fly frequently enough to know that I'm not on the list (since I don't get hassled). They also claim to have a proprietary algorithm that will do better matching.

On the other hand, on one trip my daughter (6 years old at the time) was "randomly" selected for special screening on three out of four flights.

Posted by: Joe Buck at March 14, 2007 11:46 AM


Every day I drive by the Minneapolis/St. Paul International Airport, and every day the lighted sign beside the highway informs me that the terrorism alert level is "Orange."

Posted by: Albatross at March 14, 2007 11:48 AM


You people are missing the point. The purpose of any of these algorithms is to bring back more hits than an exact match search would find. A human must then examine the results to determine if any of the names matched are the one being searched for. The point here is that s3 has a database that contains names that would be expected to be on the no-fly list. If you find your name among the possible matches, you may want to have alternate travel plans ready. It might be a good idea to jump on an open WiFi for this particular site.

Posted by: Annie Nomous at March 14, 2007 12:03 PM


Okay, I know comedy can be a weapon, but why is Jon Stewart on there? Why???!

Funny how their software doesn't pick up Saddam Hussein. I guess he's not much of a threat any more. Kind of surprising though. I assumed once they had a name on there, that person was considered suspicious and so a threat for life. Not like you can unsubscribe from the no-fly list via email...

Posted by: Sammy The Surfer at March 14, 2007 12:20 PM


Heh, I meant for the life of the database, not the person. I doubt they go through death records and remove deceased individuals who were previously threats.

Posted by: Sammy The Surfer at March 14, 2007 12:31 PM


The following names are highlighted in red, the color of terror:

John Doe
John Smith
Richard Simmons
James Kirk
George Bush

no real surprises there. Surprises are in some of the names who aren't on the list:

Boris Badenov
Dick Cheney
Peyton Manning

Posted by: Buzz at March 14, 2007 12:55 PM


I am, of course. With two exceedingly popular names, I was bound to end up on it.

Posted by: Mark J. at March 14, 2007 1:02 PM


Actuallly when referring to myself, "popular" is the wrong word. "Common" fits better. ;-)

Posted by: Mark J. at March 14, 2007 1:03 PM


Its a little disturbing that Jesus Christ and Mahatma Ghandi would get caught as baddies, but Adolf Hitler is free and clear.

Posted by: MJ at March 14, 2007 1:51 PM


I think I might be safe since my exacte match didn't occure, though my first name wasnt in the list I had several near maches, including my surname one vowal different (Johnsen/Johnson). I guess that qualifies for a second screening.

Posted by: Skippern at March 14, 2007 1:52 PM


FWIW, 'George Bush' generates more matches than 'Osama Bin Laden'.

Posted by: tim finin at March 14, 2007 2:02 PM


@ Annie Nomous

You're going to hop on open wifi to use this site, but then put in your real name? Doesn't seem so helpful.

Posted by: Michael Ash at March 14, 2007 2:08 PM


Well, 'Israel Torres' is on the list. Now you appear to be at my IP address.
When the FBI agent shows up, I'll send him (and his National Security Letter hand-written on the back of an envelope) to schneier.com to get the real IP address...

Interestingly, the names:
Ben Kenobi, Luke Skywalker and Han Solo also return hits on the list.

Some of this data must be left over from a long time ago, stored on a server far far away...

Posted by: TED Vinson at March 14, 2007 2:24 PM


Oh dear, multiple matches under various abbreviations and misspellings as well as an exact match.

Looks like I best turn myself in :-)

Posted by: Geoff Lane at March 14, 2007 2:34 PM


Fabulous. My name is "Frier." It matched "Ferrari" and "Furrow..." but not "Fryer."

Posted by: David at March 14, 2007 2:39 PM


That one director of poor films, Alan Smithee is on it too!

Also, Nancy Pelosi gets red flagged, but Vladmir Putin doesn't.

Posted by: Cowardly Lion at March 14, 2007 3:02 PM


Both "Josef Dzhugashvili" and "Josef Stalin" get a pass. Well, to their credit, his IS dead.

Posted by: Roy at March 14, 2007 5:00 PM


@Geoff
Mujahid Shaykh isn't what you think it is. I suspect it's more a title than a proper name.

"Shaykh" means "Islamic scholar" -- http://en.wikipedia.org/wiki/Sheikh

"Mujahid" is the singular form of "Mujahideen"
http://en.wikipedia.org/wiki/Mujahideen

That, and if you search for only the last name "Mujahid" it appears. Some (many) cultures address themselves with their surnames preceding their given names.

Posted by: Jim at March 14, 2007 5:18 PM


Bruce said:
> I'm not. Are you?

You weren't until you typed your name into a form on that website anyway. =)

Posted by: wjl at March 14, 2007 5:43 PM


Wait, this is a security threat. Someone could type in names to find the ones he can use in order to fly. Someone tell the FBI/CIA or whatever.

Posted by: Adam at March 14, 2007 5:54 PM


Fortunately we're safe:

From the article "Publicly available terrorist names from various reliable government and non-governmental sources were merged to create a comprehensive list."

The no-fly list clearly doesn't use reliable data sources, so the data there is unlikely to match the realy no-fly list.

Which is just as well, since my name is apparently similar enough to "Marzouk Sammour" to cause suspicion.

Posted by: Anonymous at March 14, 2007 6:29 PM


This is the most ridiculous thing I've ever seen. Unless I am totally missing the point, you type in "mike", and it highlights these 5 first names "MAS MAX MOUSA MUGIKA MUSA" -- REGARDLESS of what last name I type in.

We are looking for suspicious people, not suspicious first names and suspicious last names. Searching makes no sense whatsoever unless you search for the first and last name in the same record. ?!

Posted by: Mike 32 at March 14, 2007 7:06 PM


We need a coder in here to run a program against it that runs every combination of letters and makes a table based on "importance". Perhaps a pattern could be found that allows the creation of a name that runs clean, although it seems that random guesses are fairing pretty well here.

Posted by: jammit at March 14, 2007 10:48 PM


That algorithm is a joke... my first name (Mark) was fully matched, of course.. but for my second name it highlights one match that has only three letters equal of nine total, and misses two names later in the list that are only one letter away from the real name..

Posted by: Woo at March 15, 2007 4:42 AM


I am not surprised at all that names like "Mickey Mouse", "Jesus Christ", and "James Kirk" appear on such a list -- it is a quite reasonable example of CYA security. How likely is it that someone with honest intentions will claim that he is called "Mickey Mouse"?
And imagine what the media would do with the TSA if there would be any kind of incident in the future involving someone who was able to check in as "Darth Vader" ...

Posted by: Gregor at March 15, 2007 4:54 AM


seems i could be watched, only one of my names is on the list

Posted by: theimp at March 15, 2007 7:44 AM


I entered my real name and even though it's on the list, it didn't highlight it?

Lame.

Posted by: Ronin at March 15, 2007 9:12 AM


Interesting that all but one of S3's management team (http://s3.com/about/team.php) are flagged when run through the toy.

-CMA

Posted by: cmarnold at March 15, 2007 9:50 AM


...as well as all of Great Britain.

Posted by: cmarnold at March 15, 2007 9:53 AM


I put in Ted Kennedy, and Todd Kennett pops up red in both columns, even though Kennedy shows up below Kennett, not in red, and Ted shows up below as well, if you page down a bit. Fantastic matching. It's a wonder anyone is flying. I'm on the list, by the way, but my last name isn't red, whatever that signifies.

Next time you're in the security line, think of how much worse it would be if Richard Reid had concealed his explosives in a more proctolgical spot. Removing your shoes doesn't seem that bad now, does it?

Posted by: Chuck Emery at March 15, 2007 2:15 PM


I don't think this makes sense at all. All common first names are in red.

Posted by: Pedro at March 16, 2007 5:07 AM


"Ignignokt Mooninite" brings up red flags on first and last names. The Boston PD was right! (Err's still sneaking under the radar - because he's shorter, obviously.)

After that discovery it just devolved into me typing in names of characters and titles of Cartoon Network shows. Oh, Madame Foster, how could you?

Posted by: Archon at March 16, 2007 1:26 PM


Mysql includes a soundex algorithm now. If you have a mysql database laying around you can use the "where field sounds like 'whatever'". I think by default it gives you more than the usual 4 characters, so you can somewhat control the match accuracy.

Posted by: CR at March 20, 2007 8:12 AM


Post a comment



Real names aren't required, but please give us something to call you. Conversations among several people called "Anonymous" get too confusing.



E-mail is optional and will not be displayed on the site.


Remember Me?


Powered by Movable Type. Photo at top by Steve Woit.

Schneier.com is a personal website. Opinions expressed are not necessarily those of BT.

 
Bruce Schneier