Schneier on Security

A blog covering security and security technology.

« FIDIS on RFID Passports | Main | Cryptography Comic »

November 10, 2006

Essay on Data Mining

Good essay on data mining.

Posted on November 10, 2006 at 7:05 AM • 17 Comments

To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.

Comments

Fraud Guy • November 10, 2006 8:21 AM

Is this why Bush feels that he can label someone an enemy combatant? Because they "likely" would have caused harm.

Is this why most prosecutions of suspected terrorists have lead to convictions for non-terrorist activity, because they haven't commited such acts yet.

The "terrorists" in Florida have been portrayed as complete losers who were nowhere near able to contemplate an effective terrorist attack; were they just "found" by such a data mining method that is still being tweaked.

This is the most chilling article I have recently read.

Snag • November 10, 2006 8:59 AM

I did a quick search and couldn't find any articles on deliberately confusing data miners. It seems to me that it won't be long before someone starts smokescreening their personal information or even deliberately creating false data identities to throw investigators off track.
Again, this then leaves the innocent exposed to unwarranted intrusion.

William Meloney • November 10, 2006 9:30 AM

A while back I sent you a link to this "Privacy" rant...

http://pa-2.blogspot.com/2004/11/...

How fitting that it is actually coming out into the mainstream of 'data mining'.

PrivacyIsImportant • November 10, 2006 9:31 AM

The more I read about computer capacity and data mining, the more I use cash. Even if it means a trip to a store rather than a few minutes at my computer.

David • November 10, 2006 10:12 AM

@Fraud Guy

The guys in Florida were found through straightforward detective work, based upon a tip.

BLP • November 10, 2006 10:24 AM

@Snag:

it's harder than it looks. Adding random data doesn't take away the perhaps 2 or 3 threads they're looking for.

A little easier is to create a few identities and make sure that the various steps are being taken by different identities.

BLP

Realist • November 10, 2006 10:50 AM

@snag
"It seems to me that it won't be long before someone starts smokescreening their personal information or even deliberately creating false data identities to throw investigators off track."

You must have been off-planet for a while. People have been doing this sort of thing for a long time now -- survivalists, illegal immigrants, criminals fo all ranks, people answering surveys online, etc.l

altjira • November 10, 2006 10:58 AM

@Privacy

It's radicals like you who are messing up the system. When are you going to learn to play along?

Have you seen the new Visa check card ad? The smoothly running routine thrown out of whack by the person who uses cash? Since it's still unlikely that using cash can be banned yet,vested interests are trying to attach a stigma to it.

Snag • November 10, 2006 11:57 AM

@Realist
Not so much off the planet...just commenting on applying disinformation directly at what investigators are looking for.
Which ties into my response to BLP...
Once the patterns that count to data miners are discovered by those who want to distort their data identity, then only one or two threads need to be thrown in to throw the extrapolation. Not so much random data, but deliberate disinfo.
Trust me...as a criminal investigator of fraud I can tell you it takes the blink of an eye before the kleptotechnoratti figures the system out and beats it.
fnord How long before hackers, pranksters, libertarians, etc. deliberately begin to find ways to game the data miners? fnord
Splog anyone?

Rich • November 10, 2006 12:22 PM

@altjira
I cannot find the link, but recently I read about a store which only dealt with credit or debit cards -- no cash. And, to my surprise it is not illegal.

Steve • November 10, 2006 12:50 PM

See "The Privacy Song" by Three Dead Trolls in a Baggie.

7Null_Seven • November 10, 2006 1:25 PM

@ Rich

While United States currency is "[...]Legal Tender for all debts, public and private," that doesn't legally extend to payment for a good or service in the immediate timeframe, based on a very literal definition of the term "debt."

But if you're just now reading about stores that don't take cash, I'm really quite impressed. There are MANY businesses in high-crime areas (such as - in my experience - the south side of Chicago) that don't accept cash. This sorry state of affairs has been the status quo for at least 20 years in some neighborhoods I can think of. Having significant amounts of cash on the premises greatly increases the risk of armed robbery attempts, and results in significant expenses incurred in attempting to secure the cash, and even more when it comes time to transport the money to the bank. Before plastic became so prevalent, most dealt primarily in checks, which aren't as useful to theives as greenbacks. And I suppose that most still do. I'm willing to venture that cashier's checks are also a large part of things, even though those are more worthwhile to steal.

Being from the suburbs myself, it was quite a surprise when I walked into a KFC that had inch-thick lexan walling off the counter from the main part of the restaurant. My bank didn't have as tight a security setup as this place did. If the franchise owner thought that he could get away with a no-cash policy, I suspect he wouldn't think twice.

Rich • November 10, 2006 2:20 PM

@7Null_Seven

Wow! I guess I'm living in a smaller bubble than I realized. Thanks for the clarification.

P-Air • November 10, 2006 7:01 PM

With more and more people doing more and more of their activities online, the correlations won't need to take place w/any offline data to be meaningful. Already there's a company called AggregateKnowledge that innocently enough offers merchants a recommendation service (call it an Amazon recommendification service ;-) where all merchants agree to have their user data aggregated for purposes of providing better recommendations to their customers.

Activities around AttentionTrust may provide end-users some level of control over their info, but to the extent that they can get a free t-shirt in exchange for access, or as a requirement for making getting a discount, then I'm not sure there will be easy ways to suppress or prevent this activity. Legislation may be tough to pass since people won't be convinced of its harm.

erasmus • November 11, 2006 11:03 AM

The move towards cashless transactions that leave no trace is remorseless; very many vested interests want to see cash disappear.

And Govt isn't going to get in the way of us using cards, as it means it has more money to hand - Govt Treasuries produce a dramatically smaller %age of cash nowadays, so related instutions get more use and income from its reserves.
(Yet there are other untoward side affects; in the UK, with a high level of access to bank accounts and lots of govt systems moving to cashless payments, smaller Post Offices are becoming unsustainable)

Clive Robinson • November 12, 2006 1:53 PM

Guy Kewney mentiones in the artical that the person who told him worked for a UK company called Sainsbury's...

Well if it is the same person I think it is, they worked for British Airways before Sainsbury's. Around the time that B.A. where (succesfully) accused of stealing pasengers from Virgin...

Which B.A. did partialy by data mining and partly by tapping data out of a private data network (the full deatails of what went on have still not made it into the public domain).

I wonder if it is a coincidence or not...

supersnail • November 13, 2006 3:50 AM

Good piece.

I dont then they went far enough in there threat analiysis though and I believe this is crucial when looking at these issues.

Polititions, senior civil servants, police chiefs etc. do not feel personally threatened by terrorists. A terrosist is unlikely to get them fired or block there next promotion (quite the opposite for security hawks). Voters and particulary those who influence them are a threat. A couple of awkward media exposes and and you could find yourself in charge of a the trade delegation to Greenland.

Our leaders seem to be heading towards Stalinist world view that the people are the enemies of the people.

Post a comment

Schneier.com is a personal website. Opinions expressed are not necessarily those of BT.

Subscribe via Kindle

Blog Menu

Search

Powered by DuckDuckGo

Blog Home Page
100 Latest Comments

Archives by Date

2013 J F M A M
2012 J F M A M J J A S O N D
2011 J F M A M J J A S O N D
2010 J F M A M J J A S O N D
2009 J F M A M J J A S O N D
2008 J F M A M J J A S O N D
2007 J F M A M J J A S O N D
2006 J F M A M J J A S O N D
2005 J F M A M J J A S O N D
2004 J F M A M J J A S O N D