Bruce Schneier
Schneier on Security
A blog covering security and security technology.
« The Difficulty of Profiling Terrorists | Main | Vista Activation Cracked by Brute Force »
March 14, 2007
Find Out if You're on the "No Fly List"
I'm not. Are you?
Soundex works, generally, by removing vowels from names and then assigning numerical values to the remaining consonants.This has been the basis for the Computer Assisted Passenger Pre-Screening System (CAPPS) and it is horrendously inadequate and matches far too many names. To see just how poorly Soundex performs, visit nofly.s3.com and type in your name to assess your chances of being on the No Fly or Watch List. This is the only known publicly available site for checking your name against potential terrorist identities and databases. It was developed by S3 Matching Technologies of Austin, Texas. The company's database technicians merged the best known data on terrorists with the Soundex system to create the site.
Posted on March 14, 2007 at 7:51 AM • 60 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
Geoff • March 14, 2007 8:38 AM
One little problem with that list: one of bin laden's aliases doesn't even show up.
Do this:
goto http://www.fbi.gov/wanted/terrorists/...
Notice the alias Mujahid Shaykh
Run that one through the "watch list"
It won't come up. Anyone see a problem here or is it just me?
Wim Coenen • March 14, 2007 8:44 AM
Two matches highlighted in red.
But this appears to be just a stunt by s3 to promote their own data mining software.
Roy • March 14, 2007 8:44 AM
So to Soundex "George Bush" is a match to "Greg Bisheau"?
Wonderful work there, but I wouldn't trust them with a guest list to a kid's birthday party.
Ledow • March 14, 2007 8:47 AM
And even funnier is that John Smith is flagged... the most common name in the US or UK. :-)
Tobias • March 14, 2007 8:47 AM
I think what you all need to remember is that this doesn't flag you as a terrorist, it just flags names that could possibly from a country that could imply you need to be screened.
No Fly List • March 14, 2007 8:49 AM
And all of you that just ran your name... you are now on the list.
Geoff • March 14, 2007 8:53 AM
2 for 2
http://www.fbi.gov/wanted/terrorists/...
Abdel Muaz
Abu Muhammad
Abu Fatima
Abu al-Mu'iz
Roenigk • March 14, 2007 8:55 AM
After a misleading headline, their description states this is a "demonstration" of how the No Fly list works. The fine print even states they obtained the names used in this list from published sources (not "the" No Fly List).
They are trying to sell an improved Soundex-like algorithm, of which there are many.
Joshua • March 14, 2007 9:09 AM
Soundex? Removing vowels and assigning number values? You mean gematria?
Martin Schröder • March 14, 2007 9:15 AM
@Geoff:
Why should UBL be on it? The FBI doesn't want him for 9/11... :->
Geoff • March 14, 2007 9:21 AM
I'm trying to find this fine print that others are mentioning, but I haven't found it yet. What I have found is:
"The results generated in this demonstration are a product of a compilation of the best available data regarding suspected and known terrorists"
[Ed: I'd suspect that fbi.gov's info "should" be in there]
...
"Publicly available terrorist names from various reliable government and non-governmental sources were merged to create a comprehensive list."
...
"Like the federal government's, this list of terrorists, integrated into the Soundex software believed to be used by the TSA, is constantly updated"
From what I gather from the above it means the db is live, but they're querying it with the "Soundex" method to prove what a joke it is in order to play up their own offering (TeraMatch). However, the db (from what I'm reading in those quotes) is indeed a live db.
Stephan Samuel • March 14, 2007 9:32 AM
Traditional Moslem names, as in the kind that someone who's angry with the West would maintain, aren't constructed in the typical western firstname-lastname pattern. Someone born in Persia, Magrib, Arabia or Hindustan (they don't like our country names either) may have seven names, several of which would commonly be used, either alone or in combination, to identify them. Compound this with the fact that there is no single recognized method for transliterating Arabic into English. Then consider that sneaky people sometimes have aliases.
By the time you enter a first name and last name of a Moslem terrorist into this system, you're as likely to get a match as if you entered random characters.
I wish John was right about this being a hoax, but I fear he's not. We're 18 days early for April Fools' Day.
Neal • March 14, 2007 9:34 AM
I don't understand the list. Why all the names not shown in red? Richard Cheney. Richard is red, cheney is not. Why bother to show cheney then.
Interestingly my first and last names are red. My first, stephen, is covered by multiple red lines for variations. My middle Neal is red. My last is red. I guess that doesn't bode well for me should I ever fly. Haven't yet at 42 though so....
Well, off to Google my name and variations of terror and the like to see if I can find someone close in name.
Brianary • March 14, 2007 9:34 AM
When I wrote the Lingua::EN::MatchNames Perl module, I used Soundex as a last desperate attempt to match names, with an incredibly low confidence level. I found the Metaphone algorithm to be far, far more accurate.
Soundex sucks.
Brianary • March 14, 2007 9:47 AM
@Wim Coenen:
@Roenigk:
"...the federal government is using an algorithmic software product known as Soundex to search databases for potential terrorists."
"TeraMatch® matched with 96% accuracy compared to Soundex's 15%. Importantly, TeraMatch® only matched 4% false positives-innocent people incorrectly identified-to Soundex's 85%."
Are they trying to sell their software? Yes, but they have a very good point about the Feds using perhaps the stupidest approach available to implement their matching algorithm.
Israel Torres • March 14, 2007 10:27 AM
Nice... especially since people tying their names in also expose their IP address to the name they are searching.
Do the math.
Israel Torres
Proops • March 14, 2007 10:37 AM
Let's say your surname is Laden, and your first name is Osama Bin. That's right, you're not on the list!
If you have a non-European name, moving your surname around shouldn't be that hard.
wiredog • March 14, 2007 10:42 AM
I call shenanigans! It looks like that system mathces first names to 'suspected' first names, and last names to 'suspected' last names, but doesn't compare the first/last pair to a list of 'suspected' first/last pairs. So "John Anything" compares to, wow, quite a few... So does "anon ymous".
Anonymous • March 14, 2007 11:09 AM
@Johns,
> I believe Bruce fell for a hoax.
The preeminent security guru of our time? One of the leading lights on the internet? SURELY NOT.
Michael Ash • March 14, 2007 11:19 AM
The list is obviously broken. Osama bin Laden is on it, but so are Bob Dole, George Bush, and Al Gore.
It's a fun demonstration of Soundex, but that's all.
Only the most common names appear to be on the list.
So Maria, Mary etc are in nofly first names. Too bad if the last name also happens to be Smith or Lopez if the real database would be that plain ...
anonymous coward • March 14, 2007 11:39 AM
even "anonymous coward" is on the list :D
so half of /. posters wont ever be flying again....
Joe Buck • March 14, 2007 11:46 AM
I'm suspicious of this story; it looks like PR for a company that's trying to get the feds to buy their software. They claim, for example, that my name is a match, and I fly frequently enough to know that I'm not on the list (since I don't get hassled). They also claim to have a proprietary algorithm that will do better matching.
On the other hand, on one trip my daughter (6 years old at the time) was "randomly" selected for special screening on three out of four flights.
Albatross • March 14, 2007 11:48 AM
Every day I drive by the Minneapolis/St. Paul International Airport, and every day the lighted sign beside the highway informs me that the terrorism alert level is "Orange."
Annie Nomous • March 14, 2007 12:03 PM
You people are missing the point. The purpose of any of these algorithms is to bring back more hits than an exact match search would find. A human must then examine the results to determine if any of the names matched are the one being searched for. The point here is that s3 has a database that contains names that would be expected to be on the no-fly list. If you find your name among the possible matches, you may want to have alternate travel plans ready. It might be a good idea to jump on an open WiFi for this particular site.
Sammy The Surfer • March 14, 2007 12:20 PM
Okay, I know comedy can be a weapon, but why is Jon Stewart on there? Why???!
Funny how their software doesn't pick up Saddam Hussein. I guess he's not much of a threat any more. Kind of surprising though. I assumed once they had a name on there, that person was considered suspicious and so a threat for life. Not like you can unsubscribe from the no-fly list via email...
Sammy The Surfer • March 14, 2007 12:31 PM
Heh, I meant for the life of the database, not the person. I doubt they go through death records and remove deceased individuals who were previously threats.
Buzz • March 14, 2007 12:55 PM
The following names are highlighted in red, the color of terror:
John Doe
John Smith
Richard Simmons
James Kirk
George Bush
no real surprises there. Surprises are in some of the names who aren't on the list:
Boris Badenov
Dick Cheney
Peyton Manning
Mark J. • March 14, 2007 1:02 PM
I am, of course. With two exceedingly popular names, I was bound to end up on it.
Mark J. • March 14, 2007 1:03 PM
Actuallly when referring to myself, "popular" is the wrong word. "Common" fits better. ;-)
Its a little disturbing that Jesus Christ and Mahatma Ghandi would get caught as baddies, but Adolf Hitler is free and clear.
Skippern • March 14, 2007 1:52 PM
I think I might be safe since my exacte match didn't occure, though my first name wasnt in the list I had several near maches, including my surname one vowal different (Johnsen/Johnson). I guess that qualifies for a second screening.
tim finin • March 14, 2007 2:02 PM
FWIW, 'George Bush' generates more matches than 'Osama Bin Laden'.
Michael Ash • March 14, 2007 2:08 PM
@ Annie Nomous
You're going to hop on open wifi to use this site, but then put in your real name? Doesn't seem so helpful.
TED Vinson • March 14, 2007 2:24 PM
Well, 'Israel Torres' is on the list. Now you appear to be at my IP address.
When the FBI agent shows up, I'll send him (and his National Security Letter hand-written on the back of an envelope) to schneier.com to get the real IP address...
Interestingly, the names:
Ben Kenobi, Luke Skywalker and Han Solo also return hits on the list.
Some of this data must be left over from a long time ago, stored on a server far far away...
Geoff Lane • March 14, 2007 2:34 PM
Oh dear, multiple matches under various abbreviations and misspellings as well as an exact match.
Looks like I best turn myself in :-)
David • March 14, 2007 2:39 PM
Fabulous. My name is "Frier." It matched "Ferrari" and "Furrow..." but not "Fryer."
Cowardly Lion • March 14, 2007 3:02 PM
That one director of poor films, Alan Smithee is on it too!
Also, Nancy Pelosi gets red flagged, but Vladmir Putin doesn't.
Roy • March 14, 2007 5:00 PM
Both "Josef Dzhugashvili" and "Josef Stalin" get a pass. Well, to their credit, his IS dead.
Jim • March 14, 2007 5:18 PM
@Geoff
Mujahid Shaykh isn't what you think it is. I suspect it's more a title than a proper name.
"Shaykh" means "Islamic scholar" -- http://en.wikipedia.org/wiki/Sheikh
"Mujahid" is the singular form of "Mujahideen"
http://en.wikipedia.org/wiki/Mujahideen
That, and if you search for only the last name "Mujahid" it appears. Some (many) cultures address themselves with their surnames preceding their given names.
wjl • March 14, 2007 5:43 PM
Bruce said:
> I'm not. Are you?
You weren't until you typed your name into a form on that website anyway. =)
Adam • March 14, 2007 5:54 PM
Wait, this is a security threat. Someone could type in names to find the ones he can use in order to fly. Someone tell the FBI/CIA or whatever.
Anonymous • March 14, 2007 6:29 PM
Fortunately we're safe:
From the article "Publicly available terrorist names from various reliable government and non-governmental sources were merged to create a comprehensive list."
The no-fly list clearly doesn't use reliable data sources, so the data there is unlikely to match the realy no-fly list.
Which is just as well, since my name is apparently similar enough to "Marzouk Sammour" to cause suspicion.
Mike 32 • March 14, 2007 7:06 PM
This is the most ridiculous thing I've ever seen. Unless I am totally missing the point, you type in "mike", and it highlights these 5 first names "MAS MAX MOUSA MUGIKA MUSA" -- REGARDLESS of what last name I type in.
We are looking for suspicious people, not suspicious first names and suspicious last names. Searching makes no sense whatsoever unless you search for the first and last name in the same record. ?!
jammit • March 14, 2007 10:48 PM
We need a coder in here to run a program against it that runs every combination of letters and makes a table based on "importance". Perhaps a pattern could be found that allows the creation of a name that runs clean, although it seems that random guesses are fairing pretty well here.
Woo • March 15, 2007 4:42 AM
That algorithm is a joke... my first name (Mark) was fully matched, of course.. but for my second name it highlights one match that has only three letters equal of nine total, and misses two names later in the list that are only one letter away from the real name..
Gregor • March 15, 2007 4:54 AM
I am not surprised at all that names like "Mickey Mouse", "Jesus Christ", and "James Kirk" appear on such a list -- it is a quite reasonable example of CYA security. How likely is it that someone with honest intentions will claim that he is called "Mickey Mouse"?
And imagine what the media would do with the TSA if there would be any kind of incident in the future involving someone who was able to check in as "Darth Vader" ...
Ronin • March 15, 2007 9:12 AM
I entered my real name and even though it's on the list, it didn't highlight it?
Lame.
cmarnold • March 15, 2007 9:50 AM
Interesting that all but one of S3's management team (http://s3.com/about/team.php) are flagged when run through the toy.
-CMA
Chuck Emery • March 15, 2007 2:15 PM
I put in Ted Kennedy, and Todd Kennett pops up red in both columns, even though Kennedy shows up below Kennett, not in red, and Ted shows up below as well, if you page down a bit. Fantastic matching. It's a wonder anyone is flying. I'm on the list, by the way, but my last name isn't red, whatever that signifies.
Next time you're in the security line, think of how much worse it would be if Richard Reid had concealed his explosives in a more proctolgical spot. Removing your shoes doesn't seem that bad now, does it?
Pedro • March 16, 2007 5:07 AM
I don't think this makes sense at all. All common first names are in red.
Archon • March 16, 2007 1:26 PM
"Ignignokt Mooninite" brings up red flags on first and last names. The Boston PD was right! (Err's still sneaking under the radar - because he's shorter, obviously.)
After that discovery it just devolved into me typing in names of characters and titles of Cartoon Network shows. Oh, Madame Foster, how could you?
Mysql includes a soundex algorithm now. If you have a mysql database laying around you can use the "where field sounds like 'whatever'". I think by default it gives you more than the usual 4 characters, so you can somewhat control the match accuracy.
Charlie D'ourso • March 1, 2011 7:12 PM
I recently had to contact the customes agents who detained my german lady friend, the computer show issue, when she went on line to purchase a ticket to Maimi she contacted the airlines and they refered her to us customes in Frankfurt they finally took it off the computer. She got the ticket the second time, when arriving in the USA she was detained. I could not get a straight aner, the airlines air france said that she got her baggage and cleared after one hour of paging and no response nor did her cell phone work. I got a customes agent to check the computer and he said that she was being detained and they needed to speak to her. One hour later no response I went back to the agent who was not cooperative and said he could not give me any more informqation I ask who could help he said to find a agent with a uniform likie his as ask them. I found one and ask if he could go in and find how she is doing and what is the decision. ten minutes later my cell rang and a 407 prefix #appeared. As I answered I noticed two of his agents watching me. I could not hear him and Sandra got on the phone she said that they were sending her back to Germany the next day and would give a office room with a bede and television to stay in.
As I left the air port I thought to call his 407 number it answered to the Orlando customs office, I ask to be transfered to the Maimi airport office as I just got a call with this number. They did and the female agent aid tht they could not give me any information, I saqid just be quiet and I would give her information, that I was going to contact diplomates, news media, home land security.
The agent and air france droped the ball and gave her the green light.
I call back one hour later and a hispanic agent in a supervisoral position gave a number to call after hearing my situation.
It was late and the number did not help as it was the wrong dept they gave me another number. Agent lopez answered and he said that she would be able to get on a flight back to Germany and that might take another 24 hours I counterd with its already been 10 hours and that should be enough to process her. He informed me to check with the Air lines and when she has a seat assignment than they will get her on the plane it still may take 24 hrs. I called Air france and Deltas agent got on as they represent Air France they had a seat assignment but no ticket was issued. I called agent Lopez back and told him that I would put up my credit card if the ticket fare was a issue. He told me to get back with the airlines, I did and got to a supervisor who gave me incorrect information, that I could not purchase a ticket for her after disputing it and their less than professional response . I got back with Lopez again and it got heated, I than said that my next call will be homeland security and the German Counslet he than said that leaglly that if she had a round trip ticket which she did that the airlines had to get her back.
I contacted the German diplomat and he took down all the agents numbers names as well as the Airline info her famlies number in Germany. I said I just want to get her on the flight at 6:pm that she was book on to get back home. He after one hour the head of the German counslet call and left me a message that she would definitely be on that flight.
When they put her in the boaqrding line gave her her cell phone she called me. She said that every time that I called the would give her sit and said that my name would be placed on a no fly list and they would tap my phone. They kept interrogateing her and trying to get her to admit to a crime and acted like terriost and they kept yelling at her and saying we can keep you for a long time unless you cooperate. Once the German diplomat call the one agent tryied ince again to mid fuck her. I call the customs 407 number back and got the hispanic woman from the night before who was the most cooperative, it began to get heated I said that i knew that they were tapeing my calls and so was I she said that it was illeagle I siad only if I do not inform them . she than asked how I got all of the phen numbers thaty I reached and how did I get all my information, I said just as you are not allowed to provide me with info and I will not provied info for you. She ask how did I get the German diplomat to respon so quickly. I said that I will presue this with the fl attorney general office, homeland security and news media.
Subscribe to comments on this entry
Post a comment
Schneier.com is a personal website. Opinions expressed are not necessarily those of BT.
|
| Subscribe |
Subscribe via Kindle |
| Blog Menu |
SearchPowered by DuckDuckGoBlog Home Page
Archives by Date2013 J F M A M J
Tags  |
| Latest Book |
more books by Bruce Schneier |
Comments