How Well "See Something, Say Something" Actually Works

I've written about the "War on the Unexpected," and how normal people can't figure out what's an actual threat and what isn't:

All they know is that something makes them uneasy, usually based on fear, media hype, or just something being different.

[...]

If you ask amateurs to act as front-line security personnel, you shouldn't be surprised when you get amateur security.

Yesterday The New York Times wrote about New York City's campaign:

Now, an overview of police data relating to calls to the hot line over the past two years reveals the answer and provides a unique snapshot of post-9/11 New York, part paranoia and part well-founded caution. Indeed, no terrorists were arrested, but a wide spectrum of other activity was reported.

[...]

In all, the hot line received 8,999 calls in 2006, including calls that were transferred from 911 and the 311 help line, Mr. Browne said. They included a significant number of calls about suspicious packages, many in the transit system. Most involved backpacks, briefcases or other items accidentally left behind by their owners. None of them, Mr. Browne said, were bombs.

There were, however, 816 calls to the hot line in 2006 that were deemed serious enough to require investigation by the department's intelligence division or its joint terrorism task force with the F.B.I. Mr. Browne said that 109 of those calls had a connection to the transit system and included reports of suspicious people in tunnels and yards, and of people taking pictures of the tracks.

The hot line received many more calls in 2007, possibly because of the authority's advertising campaign, Mr. Browne said. Through early December, the counterterrorism hot line received 13,473 calls, with 644 of those meriting investigation. Of that group, 45 calls were transit related.

Then there were the 11 calls about people counting.

Mr. Browne said several callers reported seeing men clicking hand-held counting devices while riding on subway trains or waiting on platforms.

The callers said that the men appeared to be Muslims and that they seemed to be counting the number of people boarding subway trains or the number of trains passing through a station. They feared the men might be collecting data to maximize the casualties in a terror attack.

But when the police looked into the claims, they determined that the men were counting prayers with the devices, essentially a modern version of rosary beads.

None of those calls led to arrests, but several others did. At least three calls resulted in arrests for trying to sell false identification, including driver's licenses and Social Security cards. One informer told the police about a Staten Island man who was later found to have a cache of firearms. A Queens man was charged with having an illegal gun and with unlawful dealing in fireworks.

A Brooklyn man was charged with making anti-Semitic threats against his landlord and threatening to use sarin gas on him. At least two men arrested on tips from the hot line were turned over to immigration officials for deportation, Mr. Browne said.

And as long as we're on the topic, read about the couple branded as terrorists in the UK for taking photographs in a mall. And this about a rail fan being branded a terrorist for trying to film a train. (Note that the member of the train's crew was trying to incite the other passengers to do something about the filmer.) And about this Icelandic woman's experience with U.S. customs because she overstayed a visa in 1995.

And lastly, this funny piece of (I trust) fiction.

Remember that every one of these incidents requires police resources to investigate, resources that almost certainly could be better spent keeping us actually safe.

Refuse to be terrorized!

Posted on January 8, 2008 at 7:53 AM • 30 Comments

Comments

Andre LePlumeJanuary 8, 2008 8:47 AM

Something tells me those gentlemen would not think of themselves as using modernized rosary beads. :^)

derekJanuary 8, 2008 9:44 AM

It would probably be more accurate to describe Mr and Mrs Sparshott as "British expats" or "Spanish residents" rather than "a Spanish couple".

spenceJanuary 8, 2008 10:07 AM

It's funny. I always saw the ads on the subway that said over 1k people called in, and things were great because of it, but never qualified other than that one statistic. I was always suspicious if they even caught one person and why we never heard anything about it.

They play on numbers to save their own butts on supporting stupid campaigns like this. Disgusting.

There should be an anti-campaign stating those facts, the silly and benign reports people are submitting as threats.

UbermonkeyJanuary 8, 2008 10:17 AM

"Refuse to be terrorized!" - There is a fine line between paranoia and vigilance. On one hand we scream bloody murder when resources (paid for with tax dollars) are being wasted on, well, crap. On the other hand we scream bloody murder, how could we miss this if something horrible happens. I much rather waste the resources.

Also, if there were indeed calls that have lead to arrests, surveillance and what not: what are the chances of this information being disclosed to the public? Especially during an active investigation.

Brandioch ConnerJanuary 8, 2008 10:26 AM

@Ubermonkey
"Also, if there were indeed calls that have lead to arrests, surveillance and what not: what are the chances of this information being disclosed to the public?"

Considering the crap that they DO release to the press? I'd say 100%. If they found ANYTHING they'd be screaming from the rooftops that "the systems works!!!".

That the BEST they can do is this kind of crap tells you that their system is crap.

"I much rather waste the resources."

Why? All you're doing is wasting time and money and materials that COULD be better used in preventing actual threats.

Dumt RollJanuary 8, 2008 10:28 AM

I think it is important for us all to have perspective, and remember that unless we all embrace these new security measures, then the terrorists have already won.

All it takes is just one photo of a train or a trashcan, and Al Qaeda will have all the information they need to bring America to its knees.

Police should be able to shoot photographers on sight.

Nick LancasterJanuary 8, 2008 11:17 AM

The question is, would such a program *ever* produce effective/reliable results, or is it doomed to failure because of the methodology? A 'neighborhood watch' program works simply because it's much easier to recognize suspicious activity - a stranger poking around the yard, or an unfamiliar vehicle.

Stopping terrorists is considerably more difficult, and we're making the assumption that the next attack is going to involve Muslims, who will be doing something like counting commuters during rush hour - when those statistics are probably readily available in a city document, without the risk of being seen with a handheld counter (which would roll over at 1000, in most cases).

And what's to prevent the enemy from running false-flag plays? This is basic strategy - making the defense commit or guard against the wrong play.

We need defense in depth, not amateur hour.

xd0sJanuary 8, 2008 11:30 AM

@spence:
"They play on numbers to save their own butts on supporting stupid campaigns like this. Disgusting.

There should be an anti-campaign stating those facts, the silly and benign reports people are submitting as threats."

Not quite the same topic, but there is this:

http://www.youtube.com/watch?v=ka5FdP-gNF0

Bruce blogged it before, so thanks Bruce, but that was essentially an attempt to do exactly what you're saying. Stop the spin on the numbers and look at them objectively and in perspective.

The numbers cited (all I have to go by on this one) show a 9.1% (816/8999) investigate rate in 2006 and a 4.8% (316/13473) investigation rate in 2007. If I was reviewing this I'd say they got worse not better in 2007, due to our ad campaign creating noise (less calls worthy of investigation, but more overall calls).

So the net result of the campaign's being advertised was more paranoia, less effective use of resources, and an overall reduction of the effectiveness of the program. They did however get citizens to report on other citizens enough to catch some minor crimes, so I guess that's a win in their eyes.

bobJanuary 8, 2008 11:44 AM

You have to be able to read political output properly - the best result they show in this is some "arrests"? Thats like giving someone a medal because when he shot himself in the foot he didnt hit anybody else at the same time. People get arrested to keep them from fleeing and prevent them from compromising evidence during an investigation. If you want to impress me, show me "prosecutions" or better yet "convictions". Arrests (but nothing further) means these people WEREN'T actually doing anything wrong.

And is a "cache" of firearms unlawful? is that what would be called a "collection" but for being possessed by someone you dont like? How many firearms does it involve? 3? 218?

Its like the politics of the Brady law. The Clinton camp loved to spout statistics like "40,000 people were denied firearms purchases under the Brady law this year". But if they had been JUSTIFIED in depriving these people of their civil rights, then why were there only like 27 people actually PROSECUTED under that law (since filing a false application for a purchase was in itself a felony).

What that means is that it had a 99.93% false positive rate (or a .07% success rate); or as a security program it was somewhat less successful than having astronauts on the ISS deorbit baseball sized titanium spheres by throwing them manually against the direction of orbit and hoping that when it reentered the atmosphere months or years later it landed on someone and that person happened to be a criminal.

AlanJanuary 8, 2008 11:53 AM

How does one measure the success of a preventative control? Might the visibility of SSSS have had some effect?

I've been to the NYPD command center and seen how this data is used. It also goes contributes to existing investigations. What's absent are the statistics on how the data is used in cases that were already under investigation.

Michael DuckerJanuary 8, 2008 12:06 PM

It gets better. At the end of the article, they note that the campaign cost $3 million. Of the 18 arrests made in the campaign, 5 were people who had called in false tips to the campaign - I discount those since the campaign probably caused those.

That leaves $3 million spent for 13 arrests.. or $230,769 spent per individual. Think of how many neighbourhood beat officers, or trained subway officers New York could have been bought with that money. And how much more effective they would have been.

Refuse to be terrorised is right, sir.

AnonymousJanuary 8, 2008 12:10 PM

A count of arrests is useless unless they translate to charges, prosecutions, convictions and jail time.

We see the same crap in the UK, where the security forces regularly leak the number of people they are "watching", yet exactly none of the supposed conspirators are ever arrested and charged.

jblJanuary 8, 2008 12:14 PM

@xdos: "They did however get citizens to report on other citizens enough to catch some minor crimes, so I guess that's a win in their eyes."

Similar to the oppressive elements of the horribly "acronym'ed" USA PATRIOT act, the tools declared to be necessary to fight "terrorism" have been most successful in bringing about more criminal arrests. We have been given no visible proof that any added security was provided worth either the money we have spent or the civil rights we have "spent".

AlanJanuary 8, 2008 12:34 PM

I keep wondering what threat photographs could be used for in the first place. I try to look at it as if I was trying to plan an attack. What would the photo tell me that could not be accomplished by Google Maps or Mapquest?

The only thing I can think of is that the people who are trying to "protect us" are firm believers in Black Magic. They believe that if they have pictures of the object, they could somehow attack the object at a distance. (Maybe with prayers and pins or something.) Having a picture of an objects steals some of the essence of the object (or "copyright"). If it loses enough essence, it loses any divine protection it may have and could be reproduced at will.

Doctor Strangelove has nothing on the DHS and TSA.

jkcJanuary 8, 2008 2:28 PM

It's clear that in this case, a false negative is far more expensive than a false positive. Thus it's appropriate to have what might seem at first blush to be an excessive number of false positives.

How many false positives are too many? That requires quantitative study of a lot of imponderables, not least of which is the economic value of a human life. Once you settle on a figure for the economic cost of a terrorist attack, though, you have something concrete by which to evaluate the cost of prevention - not just a gut feeling of "My God, look at all those false positives!".

The efficacy of investigation is a separate issue from the problem of instilling hysterical fear in the public for political gain. It's the latter that needs to be directly addressed; if that problem is solved, the volume of false positive reports will in turn fall dramatically.

bobechsJanuary 8, 2008 4:00 PM

@ Anonymous- whichever ye may be.

And what makes you think the terrorists won't first begin probing security by pedestrianating on the bike paths, seeing what happens?

And as another anonymous observed here recently- these are CRIMINALS we are talking about here. How can you be on their side??

SnarkJanuary 8, 2008 10:11 PM

I phoned in a tip to the homeland-security hotline. I'd been acting suspiciously lately. I'm probably harmless, but you can't always tell. I could be anyone.

aracneJanuary 9, 2008 3:08 AM

I'm mystified by the "the terrorists are counting people" reasoning.

I can't even reach that degree of irrationality. Is bomb placing such a fine art that you need to count if they are 2'1 or 2'3 persons per square meter? With an electronic device? calling attention to yourself?

nycphotorightsJanuary 9, 2008 6:48 AM

What is even more frightening is that authorities are using these calls to do an end run around the law and effectively nullify rights granted to us by the constitution and by specific laws:

Proposed MOFTB regulations allow filming and photography in the City of New York without a permit from public property as long as you do not block the entire sidewalk - 8 feet must be left passable.

Photography in the subway system is specifically permitted by 21 NYCRR (New York Code Rules and Regulations) 1050.9 (c).

The harassment of photographers nationwide is being tracked by www.nycphotorights.com. It is outrageous that the authorities are using security as the excuse to engage in a concerted effort to destroy a legal hobby.

CGomezJanuary 9, 2008 8:54 AM

An earlier commenter nailed it. When something bad finally does happen, witch hunts are the order of the day. People lose their careers and their families lose everything because of the media's need to convict people in the court of public opinion.

So I don't blame authorities for being quick to investigate anything. Better to cover yourself then lose your livelihood in the subsequent Congressional hearings about how you should have known better.

The public attitude has to change. Officials who conduct witch hunts have to be thrown out of office. Our elected officials are a reflection of ourselves. We tacticly approve of such behavior.

Refusing to be terrorized has nothing to do with it, because as a society we approve of gutting people's careers when they merely happened to be on watch when something bad happened... even if they were doing their job just fine.

Refuse to be finger-pointers is probably a better all-around solution.

LyleJanuary 9, 2008 10:21 AM

"They did however get citizens to report on other citizens enough to catch some minor crimes, so I guess that's a win in their eyes."

It's not even clear they did that. The statistic includes calls transferred from 911 and 311, so we have no idea how many of those reports would have been made in the absence of this campaign. Some, no doubt.

AnonymousJanuary 9, 2008 12:06 PM

People talk about catching civil crimes with these security measures as if that's a good thingand makes up for not catching terrorists. Consider the cost of the campaign, though. If you actually did something with that money that was _targeted_ at catching civil crimes, we'd probably catch far, far more. Or maybe even threw the money at investigations of actual crimes rather than potential ones? We might solve some of them.

DavidJanuary 9, 2008 8:12 PM

@CGomez:

Okay, so how many people were burned in the witch-hunt over the September 11 attacks? As I remember, the people primarily responsible for US security all kept their nice comfy jobs. How many people were held accountable for the truck bombings in Beirut way back when? President Reagan said he accepted all responsibility, and didn't have the honor to resign when he said that.

It appears that screwing up big-time and allowing terrorist attacks with a big death toll isn't a problem, and all this talk of witch-hunting is a farce. Nobody is held accountable when things go wrong, whether they screwed up or not.

Maybe we should have some witch hunts after things like the 1991 attacks. You know, like Admiral Kimmel was relieved after the Pearl Harbor attack. It might give people the idea that high office actually comes with responsibilities.

Peter GalbavyJanuary 10, 2008 11:36 AM

I read the linked story about "young white Icelandic woman" etc.

Does anyone think that a "young/old African/Middle eastern/Chinese man/woman" would suffer less humiliation or embarassment in similar circumstances ?

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of Resilient Systems, Inc.