CIA Invests in Social-Network Datamining

From Wired:

In-Q-Tel, the investment arm of the CIA and the wider intelligence community, is putting cash into Visible Technologies, a software firm that specializes in monitoring social media. It's part of a larger movement within the spy services to get better at using "open source intelligence" -- information that's publicly available, but often hidden in the flood of TV shows, newspaper articles, blog posts, online videos and radio reports generated every day.

Here's the Visible Technologies press release on the funding.

Tags: , , , , ,

Posted on October 26, 2009 at 6:53 AM • 36 Comments

Comments

Vincent • October 26, 2009 8:19 AM

Wired's quotation of the term "open source intelligence" as though it were a neologism or a novel idea reflects some misunderstanding of the term and of what the agencies have always done. From the anecdotes of officers, I've gathered that a sizable portion of the nation's intelligence gathering resources have always been devoted to open sources. You can learn a lot by reading a newspaper.

Anyhow, you're going to hope that the intel services aren't investing any more in the shallow, novel, sources than warranted. Social networks can be gamed. Entirely false and largely fabricated identities are constructed all the time for all sorts of reasons.

Schneier, Bruce • October 26, 2009 8:45 AM

@ Vincent:

That is completely untrue what you say about false and fabricated identities.

The gall it must take to even suggest such a thing.

Schneier, Bruce • October 26, 2009 9:01 AM

I think the second post comes from a fake Bruce :) Bruce's posts appear usually highlighted in yellow.


P.S. I am just another Bruce Schneier

Patrick G. • October 26, 2009 9:22 AM

And some people said there is no way to earn Big Money with Twitter, Facebook and all the other "Social" sites.

Now there is!

And even cleverer if you sell your data (or data mining software) to agencies who can ignore all privacy and data protection issues.

But hey, we don't have anything to hide, do we?

wiredog • October 26, 2009 9:26 AM

@Patrick G
If you have something to hide, don't put it on twitter, facebook, or any other place it might be linked with your real world identity. This applies not just to hiding from the gov, but also to hiding it from potential employers, online family members, etc.

Louis • October 26, 2009 9:33 AM

@wiredog

The best way to put it, and I don't remember the source, is that you only post stuff on Facebook if you're ready to announce it to a full capacity stadium.

Securitt • October 26, 2009 9:39 AM

I am sure that there are lots of things happening in Social networks that we are unaware of.

Some of which probably included terrorism and anti-gov movements.

This is probably what they are really looking for, no criminal wants to host there own communications.

They would rather offload it onto someone elses infrastructure. That is the smart thing to do and if you communicate the lingo in another covert channel one time, then from then on communications can take place and no one is the wiser.

wiredog • October 26, 2009 9:54 AM

@Louis.
Yep. Or any other online site. Like this one, slashdot, and every other place where I'm "wiredog".

Vincent • October 26, 2009 9:57 AM

@Securitt
Analysts have always read what's in the press, but with so many journalists now out of work and blogging freelance, paid or unpaid, many open sources have been shut down or have moved into other publishing paradigms. There's some argument for data mining, but you've got the larger issue of data mining /credible sources/ that isn't really addressed in these commercial packages.

When you have to evaluate a source's credibility anyway, there isn't really any suffering from too much information that can be alleviated by a data mining tool.

Terrorism and anti-government movements aren't usually quite so well hidden, and at any rate it would probably be silly to collaborate in any sort of anti-government conspiracy on darpanet. The feds are pretty savvy about this stuff.

satan • October 26, 2009 10:13 AM

As glamorous and dramatic as this CIA stuff may sound, most of the readers here should probably be paying more attention to the sort of work that I'm doing right now. I'm sifting through data collected on spending and referral habits in facebook apps with opportunities for micropayments to figure out which users can be milked for cash (and how much). I'm also suggesting that for user statistics which correlate strongly to monetization, experiments should be done to incentivize the higher-spending behaviour and see if the relationship is in fact causal. If it's causal, then the clients will know what to do without my having to tell them.

More briefly, I'm turning people's crackbook into their methbook.

The CIA doesn't care about you. Neither do I really, but the CIA will ignore you because it doesn't care. Because I don't care, I won't think twice about bankrupting you with a micropayment addiction.

Have a nice day.

David • October 26, 2009 10:40 AM

@wiredog: Bear in mind that posting as "wiredog" isn't going to stop good investigators (and the CIA has some) from finding out who you really are. Don't put anything on Facebook or Slashdot or blog comments if you're not willing to have it broadcast to the world under your real name.

Just a little extra paranoia.

moo • October 26, 2009 10:51 AM

This is mildly interesting, when considered next to the persistent rumors (several years old now) that Facebook itself was at least partially funded by an investor with ties to the CIA.

I've always assumed there was some truth to it, because where's the downside for them? Facebook is just another source for them to datamine, however, it has an unusually rich concentration of associations between individuals, personal messages, etc. Facebook was initially just for college/university students and business professionals; two groups likely to give rise to radical politics (and thus, to be worth keeping an eye on). Then as its popularity grew, it started to let in high-school students and then just everybody. If Facebook users are happy to put their entire lives and all of their personal relationships into an online database, why wouldn't the CIA just datamine it for every scrap of data it could?

Even if most of the data is not useful now, they might find something in there which is useful in the future. There are lots of potential uses, even legitimate and benign ones (for example, if they have a terrorism suspect 5 years from now, they might find it useful to comb through archived facebook data from several years earlier to help them discover old college acquaintances of the suspect, who they could then interview to help build a profile of him).

Clive Robinson • October 26, 2009 10:52 AM

@ Satan,

"with opportunities for micropayments to figure out which users can be milked for cash (and how much)."

Out of interest which micropayment system are you looking at?

The reason for my interest is I used to work for a company called Silverplatter and back in the 90's I looked into payment methods (not other forms of revenue such as advertising) of which their are three basic types,

1, Subscription.
2, Pay-as-you-go.
3, Micropayments.

At the time I realised that subscription income would probably always be less than 5% of the available income and a "cut-throat" market.

Pay-as-you-go needed a very low overhead administration system but was workable with then available payment systems. It would be worth about 10-15% of the available income.

However micropayments for "grazzing access" would with the correct systems inplace represent upto 80% of the available income. At the time there where no realy viable micropayment systems the best examples would have been phone operators.

It's been well over ten years since I last looked and I'd be interested to hear about micropayment systems that people consider viable.

altjira • October 26, 2009 11:11 AM

Didn't some researchers report a very high success rate figuring out whether someone was gay, based on who they were friended with on social sites? Not that I give a damn, and it hard to believe that anybody else still does, but if they can do that, surely the CIAand others can keep tabs on suspects based on whom they associate with. Hell, they can probably predict when someone is going to go bad even before the subject realizes it.

D • October 26, 2009 11:30 AM

"Open source" is the new "e-". I was at an art show the other day for, I kid you not, open source embroidery.

I guess I should just accept our new open source world. I should just sit back in my open source chair and sip on my open source coffee. Perhaps I will be more content if I just watch the open source clouds float by and the pedestrians walk down the open source sidewalk.

Mammon • October 26, 2009 11:31 AM

@Clive Robinson:

I can't speak for the prince of darkness, but the most successful existing micropayment systems I'm aware of are indeed based on selling telephony by the minute in a tranferable form (international calling cards, prepaid cellphone time) and then allowing people who receive the transfers to spend them on things other than telephone access. The biggest issues with accepting these payments are, firstly, that you have to associate yourself with obvious chiselers to the detriment of your own public image, and secondly, that you have to associate yourself with obvious chiselers to the detriment of your own share of the profit.

kashmarek • October 26, 2009 12:03 PM

You are overlooking the obvious. Since there was a press release about this datamining activity, that is what "they" wanted you to look at. What "they" are really watching is for the change in behavior due to the press release, and then "they" will know who to watch.

Like hunting game birds, walk the corn field and the birds move to the bean field. Walk the bean field and they move to the pasture next to the cows.

HJohn • October 26, 2009 12:29 PM

@kashmarek: You are overlooking the obvious. Since there was a press release about this datamining activity, that is what "they" wanted you to look at. What "they" are really watching is for the change in behavior due to the press release, and then "they" will know who to watch.
________________

That may be the case. I once participated in an interesting audit where we ran a backup then passed word of the audit around, then we ran another backup and looked at what was deleted between the two. Basically, this showed us what they didn't want us to see.

Vincent • October 26, 2009 1:29 PM

Incidentally, I believe that the term "open source" existed as a noun phrase in the intelligence community long before it was an adjective and verb phrase in the software community.

Clive Robinson • October 26, 2009 1:29 PM

@ D,

"I should just sit back in my open source chair and sip on my open source coffee."

Tut tut, unless you are "managment" it should be Ubuntu / Linux Cola.

I'm told that one's "fair trade" and the other "Open Source".

John Campbell • October 26, 2009 1:31 PM

Well...

Social networking is where the least-censored "news" gets posted, mostly from the POV of an observer. Consider Twitter... there may be some value.

A lot of the traffic will be, of course, noise, but, with some slight amount of "intelligence" (usually a word that CANNOT be used for *ANY* governmental agency unless one is projecting either parody or satire) it can give some early warning of "real news".

It is NOT like conspirators are going to use these technologies as a communications medium-- thus making it USELESS for COMINT-- but, really, CNN and FOX, if they can get off their high horses w/r/t "being REAL news", would be eager to mine these kinds of sources for the next great scoop.

News is all about anomalies... and, given a baseline "flow" through these resources... there will be notables DESPITE the fact that the USA contains a polymemetic society.

RH • October 26, 2009 2:18 PM

@ wiredog's "If you have something to hide, don't put it on twitter, facebook,..."

Of course this also ignores that others freely give away your information for you. When I finally broke down and got on facebook, My name alone was enough for it to sugest everything from an old ex-girlfriend to a teacher from college I took one course from. My only theory is that they gave facebook their address book and it included me!

le Chao • October 26, 2009 2:20 PM

@Clive:
I didn't catch your metaphor with Linux and cape a cofee. Linux BETTER becouse it's Open Source as well as IBM and Sun sponcored.

le Chao • October 26, 2009 2:24 PM

@RH:
In our country this information could be bought from governent database administrators. So what?

cipherpunk • October 26, 2009 2:33 PM

Despite these articles presented to the public, how many social networking user's will actually heed the warnings; very few if you ask me.

I think what we must pay attention to is how the government chooses to define "open source intelligence". Of course services like Facebook, Myspace, and twitter to name a few, by default the posts are published and viewable by virtually anyone. When a user chooses to minimize who can view their postings either by making it friends only or enabling any other privacy setting, in my opinion the information they post can no longer be classified as "open source"; what happens if the government is still able to mine this data, by whatever means? How will they justify that? Possibly draft up and pass some new legislation that can justify their sidestepping.

The simplest thing to do is limit one's exposure online.

AppSec • October 26, 2009 3:12 PM

I'm less concerned with the Government mining this data, then I am about people who I work with. The odds are higher that a person at my current employer can do more damage to me then the government actually caring about anything I have to say.

My guess is that holds true for the majority.

Clive Robinson • October 26, 2009 6:39 PM

@ le Chao,

"I didn't catch your metaphor with Linux and cape a cofee."

Supposadly "programers survive on pizza and high cafine cola", "managers survive on coffee and chewing off the heads of their subordinates".

I'm not sure if it is real or not but supposadly there is a "fair trade" soft drink called "Ubuntu cola".

And likewise an "Open Source" software project called "Linux Cola".

Google both colas for more info and pictures.

Clive Robinson • October 27, 2009 12:15 AM

@ cipherpunk,

"Despite these articles presented to the public, how many social networking user's will actually heed the warnings; very few if you ask me."

At this time only those considered vaguely "paranoid" by their friends.

However as time goes on and more people get hurt by such things people will start to be more cautious.

Have you ever thought how technology is now driving evolutionary change?

There is evedence that it is true. In certain parts of Europe wolves no longer howl at night to attract mates etc...

The reason has been attributed to the fact it draws attention to the wolf that gets rewarded by being shot before it gets the chance to mate.

Those that don't howl get to pass on their genetics so the next generation is less likely to howl.

The same with wolves predating farm live stock.

It has got to the point where people have wolves living around them and the only real indication is signs such as "spraints with fur in" and occasional track.

This is all due to the simple technology of the gun...

What will the technology of DBs and mass survalance do to our children and grand children, and theirs in turn?

Calum • October 27, 2009 5:59 AM

@Clive, indeed there is an Ubuntu cola. They sell it in my University canteen.

Terry Pratchett put it best when it comes to evolution: there's no point evolving slowly. When you fall off a cliff, you need to evolve wings pretty damned fast.

Hedgehogs are another example: they've learned to not curl up into a ball when a car approaches, and instead they leg it. Flat hedgehogs are now, happily, much less common than they once were.

Clive Robinson • October 27, 2009 10:15 AM

@

"there is an Ubuntu cola. They sell it in my University canteen."

Is it any good?

And if it is does it say who makes / distributes it on the can?

Mind you speaking of comestables I guess the old gag about hedgehog sandwich and "real lorry drivers" no longer applies then?

Jokes aside for some unacountable reason I have a real soft spot for hedgehogs and feed my local ones cat food and thankfully unlike last year they are looking healthely plump this year.

Charles N. Steele • October 27, 2009 11:21 PM

The more information collected, the more the task of using it approaches impossible. If one is really paranoid about the government monitoring one's communications, put everything into Pashto or Farsi, and while it will surely be collected, there's almost no chance it will ever be translated, much less analyzed.

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

← Friday Squid Blogging: Steampunk Squid Cake 2006 Wal-Mart Hack →

Sidebar photo of Bruce Schneier by Joe MacInnis.