Wanted: Trust Detector

It’s good to dream:

IARPA’s five-year plan aims to design experiments that can measure trust with high certainty—a tricky proposition for a psychological study. Developing such experimental protocols could prove very useful for assessing levels of trust within one-on-one talks, or even during group interactions.

A second part of the IARPA proposal might involve using new types of sensors and software to gauge human facial, language or body signals that might help predict trustworthiness. Perhaps facial recognition technology that could deduce emotions or facial tics might help, not to mention better lie detectors.

IARPA is the Intelligence Advanced Research Projects Activity, the U.S. intelligence community’s answer to DARPA.

Tags: intelligence, psychology of security, sensors, trust

Posted on March 11, 2010 at 6:17 AM • 44 Comments

Comments

anonymous coward • March 11, 2010 6:40 AM

Yet another agency. As if you yanks could actually afford it. Ur broke!

BF Skinner • March 11, 2010 6:45 AM

The Voight-Kamp test, again? Either invent the damn thing or discard it for good.

@anonymous coward “Ur broke”
Don’t you believe it for a minute AC. It’s a matter of priorities. If you don’t think people should get assistance at all then no there’s never been enough money. But oddly there’s always enough for a new boomer.

CuriousGuru • March 11, 2010 7:04 AM

My cheapskate prediction is that this will fail to generate anything significant or useful.

Good luck to them though…. Any spare funding and I will measure specific gravity of empty beer glasses.

Roy • March 11, 2010 7:49 AM

They seek “to conduct high-risk, high-payoff research” ?

If it is high-risk, it is unlikely to pay off at all. If it is sure to be high-payoff, it has to be low-risk.

They’re kidding themselves, of course, but the intent is to fool the grantors into forking over the funding.

Unconvinced • March 11, 2010 8:08 AM

Maybe I’m reading too much into this, but I find the premise vaguely Orwellian. (Substitute “loyalty” for “trust” and say it out loud a few times.) Imagine the airport screening line scene: “I’m sorry, ma’am, but our algorithm has decided we shouldn’t trust you. Please step this way.”

The question isn’t the developers’ intent, which I’m sure is good. But technology is ethically neutral in its application — and this is one that if it works has a whole range of dystopian end-uses in the wrong hands. That’s true of a lot of things, but with the enthusiasm for deploying large-scale surveillance and screening techniques in the last decade, you have to wonder.

spaceman spiff • March 11, 2010 8:46 AM

Sounds like a winner for cheating at poker…

John Campbell • March 11, 2010 8:46 AM

Well, when it comes to surveillance, lookit this:

http://yro.slashdot.org/article.pl?sid=10/03/10/2354209

The Apple iPhone (and iBook) showed that adding accelerometers was no big deal and now it looks like a good lead-in to additional surveillance.

The very feature that allows you to make light-saber noises with an iPhone ALSO allows someone to track your activity. If this doesn’t prove that multiple uses for a “neutral” technology can’t be turned to authoritarian uses…

Carlo Graziani • March 11, 2010 8:46 AM

Must-resist-urge-to-post-yet-another-screed-about-the-Feds’-magical-thinking-and-polygraphing…

xman • March 11, 2010 8:57 AM

lets see, dmso and botox for a quick spray on product, yeah, I could make that. Just for poker night.

Beta • March 11, 2010 9:11 AM

@Unconvinced: “Maybe I’m reading too much into this, but I find the premise vaguely Orwellian. ”

Nothing vague about it. From 1984: “The scientist of today is either a mixture of psychologist and inquisitor, studying with real ordinary minuteness the meaning of facial expressions, gestures, and tones of voice…”

These research projects have gotten a lot of funding over the years and have never lived up to their promises (and I hope they never will). The results never stand up under scrutiny, but as long as they’re kept hidden they can seem impressive.

It occurs to me that if such a method were disclosed to the public and it actually worked, it could be applied to political speeches…

Larry • March 11, 2010 9:28 AM

Agree with Beta. The real trick is to make sure any trust metrics, if they’re applied at all, are applied equally and across the board. If the politicians and security folks want to know of they can trust us, fine — we want to know if we can trust them, too.

If it works, and we can point it at them, then frankly I think we all stand to benefit.

The problem, of course, is that it’s hard to get right, and it’s hard to “point it at them”. Have you ever asked a telemarketer for their home phone number? Or followed a police officer home? Or even just filmed them doing their job in broad daylight? Oddly enough, a lot of them don’t like it.

(As an aside, James Halperin explored some of these ideas in his novel The Truth Machine.)

HJohn • March 11, 2010 9:31 AM

False positives could potentially be disasterous. Not only an abomination to liberties, but a discrimination lawsuit waiting to happen.

RH • March 11, 2010 9:53 AM

Clearly they should take this to Linsay Lohan. She clearly doesn’t trust E*TRADE one lick!

The flip side is “what if we could measure trust” (which is what the title says, even though the article is about a trustworthiness tester). It would be a tool for determining how badly you can con someone!

Carlo Graziani • March 11, 2010 9:56 AM

Argh. OK, sorry, I have to.

Look, the “Orwellian” aspect is irrelevant. This stuff doesn’t work, in the sense that government and vendors are super-careful to keep these sorts of behavioral techno-metrics away from any kind of peer-reviewed assessments. That way, the Federal securocracy can continue to indulge its techno-mystical fantasies of automated loyalty screening, and the vendors can cheerfuly make fatuous claims about system reliability.

When such systems are scientifically reviewed (such as when Congress mandated the 2003 NAS study of the effectiveness of polygraphing, over DOE’s objections), the findings have been that the response curves (true positive rate as a function of settable false positive rate) never possess “sweet spots” (settings at which the true positive probability is high for a low false positive probability). That is to say, as loyalty screening tests, they are useless.

Needless to say, this has not even slightly dented the Fed’s enthusiasm for polygraphing. It has only taught them — and their snake-oil vendors — that demands for genuine scientific assessment of these sorts of systems are to be vigourously fended off, or when that is impossible, the assessments themselves are to be simply ignored.

Given that attitude, it is utterly clear that there never will be any genuine scientific basis for asserting that loyalty/trust/evildoing/badthink detection technology adopted by the Federal government actually performs as advertised. Given how hard this sort of thing is to do even crudely, that kind of intellectually corrupted incentive system is tantamount to a guarantee that it will never work. The Feds will go on performing their periodic mystical hazing rituals using shiny polygraph vendor brochures as justification, and the results will continue to be as reliable as they ever were.

kashmarek • March 11, 2010 9:57 AM

Terms such as facial, one on one, voice, body language, all speak as though you were able to interact with the other person (you know, the terrorist, or the individual using your stolen identity, or using fake information on an internet transaction).

a different phil • March 11, 2010 10:00 AM

What’s that old saying? “Once you learn how to fake sincerity, you’ve got it made.”

jgreco • March 11, 2010 10:07 AM

@BF Skinner

“The Voight-Kamp test, again? Either invent the damn thing or discard it for good.”

Yeah, though the first thing I thought of was the pervasive monitoring and regular polygraph testing of government employees in Snow Crash. That’s how I picture this sort of thing being used anyways, sounds like the actual mechanics of how it works is closer to a Voight-Kamp test. I guess life really does imitate art…. unfortunetly.

DayOwl • March 11, 2010 10:58 AM

Three elements of the Bureaucratic Holy Grail in one package:

Eliminate all risk.
Tools that allow us to arbitrarily exclude people without our having to defend it.
No responsibility whatsoever for decisions.

Magical thinking in action.

BF Skinner • March 11, 2010 11:00 AM

@jgreco “imitate art”
What I liked about PKDs version of the VK was that it couldn’t tell the difference between human and non-human. those type I and II errors. pesky. Tsk.

@Unconvinced “Substitute “loyalty” for “trust””
That’s not a leap. McCarthy, Angleton, and now Liz Cheney (and Dad and other previous administration officials rehabilitating themselves) demanding DoJ “prove” their career lawyers aren’t disloyal scum because they insist on the rule rule of law.

HJohn • March 11, 2010 11:08 AM

@BF Skinner: “That’s not a leap. McCarthy, Angleton, and now Liz Cheney (and Dad and other previous administration officials rehabilitating themselves) demanding DoJ “prove” their career lawyers aren’t disloyal scum because they insist on the rule rule of law.”

He didn’t make a leap, but you sure did.

BF Skinner • March 11, 2010 12:15 PM

@HJohn “He didn’t make a leap”

Give them a tool for purpose A and get purely predictable B.

The former Administration took a wish list off DoJs shelf and passed it. President Obama has decided they like the Patriot Act so much it should continue. I’d be interested in seeing how the data on how much the financial provisions haven’t caught terrorists but have caught tax cheats.

I recently got to witness a TSA training session. The trainer was teaching screeners how to id white powders on the xray. Not explosives – cocaine.

TSA airport checkpoints are routinely staffed, “assisted” is the word they use, with other federal agencies (USSS, CBP, ICE, DEA) and local law enforcement to “leverage and support each others mission.”

So just how do drugs affect the safety of the Aircraft, crew and passengers? I posit the drugs are outside the bodies of the crew.

The TSA checkpoints have become internal Checkpoint Charlies. We’re not talking international borders here. We are searching (soon to be semi-automated strip searches) 10s of thousands of people every day that fly from one domestic airport to another. In the name of “safety” but in reality the scope has creeped and any suspected crime is being pursued.

Petréa Mitchell • March 11, 2010 12:36 PM

I think it’s just as well this is impossible. A completely trustworthy person can still be wrong. What would really be handy is a magic system that can tell you which information to trust, instead.

Clive Robinson • March 11, 2010 12:47 PM

@ HJohn, BF Skinner,

Speaking of “unwarented creep of powers”.

In the UK we have a problem with “gang culture” in that there is a very solid corelation between the shuting down of “youth amenaties” and the rise of “gang culture”.

The “liberal laws” have aded and abeted the rise of “youth terroristic” activities in that they know that the law has few penalties for them so shooting, stabing, rape and even murder are routien activities for gang members to get their “colours”.

So the Gov bring out laws to crack down on legaly owned guns, illegal gun crime goes up. Likewise they crack down on legal knives, and illegal knife crime goes up.

The latest Government wheeze is to make dog owners get compulsary insurance at around 1000USD per dog per year just to stop the rise of dangerous dogs that happened in response to the “dangerous dogs act”

All the time youth gang culture and the attendant crime continues to spiral upwards whilst the police run around confiscating small pocket knives simply by pretending they are going to be used for a crime.

If there is ever a wrong way to go about dealing with a serious issue you can be sure a politician is going to go that way, to the compleate detriment of the general population.

Judging by the figures if we take all the money wasted on these idiotic policies we would get rid of the national debt within a couple of years not the two to three generations that some economists are predicting.

I just wish there was some way of getting rid of the incompetent self serveing ingratiates. Who blaim the public when their greed and incompetence is unmasked for all to see.

Clive Robinson • March 11, 2010 12:54 PM

Anybody read “Dune” and remember about “Imperial Conditioning” that was supposed to be an unbreakable way of getting loyalty out of an individual?

And how (as normal) somebody worked out a way of breaking the system and thus getting a trusted person to turn on their “master” and betray them to an enemy…

The point being you might be 100% trustworthy today but what about tomorow?

Trustworthyness is like the toss of a coin previous performance is no indicator of future performance.

Ian • March 11, 2010 2:39 PM

@Clive

I always thought it was lame that Herbert couldn’t come up with a better subversion method for that. What, nobody takes hostages in the future? Good book, otherwise.

I do agree with your point, though. How often would people get re-tested to ensure continuing loyalty? Not only that, but how would this system give them any more useful information about a person’s potential for betrayal? I hold a US Gov’t clearance and have to go through periodic lectures from the Defense Security Service guys who never fail to remind us that most clearances aren’t lost for some ideological reason – it’s mostly financial. Bob has access to sensitive info and is about to lose his house to foreclosure. Wife has cancer, kids need braces, credit cards are maxed out. Some nice fella offers to pay the bills if Bob leaks some relatively harmless info. Even the most loyal public servant/soldier would be tempted to give it, and that’s why they carefully review your financials when getting/renewing a clearance. How would this “trustworthiness” test add anything to that? What would it tell you that other checks wouldn’t?

HJohn • March 11, 2010 2:51 PM

@Clive: “Trustworthyness is like the toss of a coin previous performance is no indicator of future performance.”

I tend to agree, although I’d modify it say “previous performance is no guarantee of future performance.” I believe their is a correlation between past and future, but it’s not a guarantee.

I’ve seen once trusted people turn when their circumstance change coupled with opportunity (ironically, opportunity may be there because of their track record of trustworthiness). Financial ruin, a bad divorce, a nasty custody battle, etc.

I also firmly believe that if one tries to measure trusts through structured means, people will find ways to appear trusted. Looking the part and acting the part goes a long way to avoiding scrutiny and detection.

mcb • March 11, 2010 3:24 PM

@ clive robertson

“Anybody read ‘Dune’ and remember about ‘Imperial Conditioning’…?”

“Yueh! Yueh! Yueh! A million deaths were not enough for Yueh!”

JimFive • March 11, 2010 3:49 PM

re: Dune

The irony of Dr. Yueh’s betrayal isn’t that no one took hostages it is that no one loved. The Dune universe was one in which it was inconceivable for one person to love or even trust another without some sort of hold on them. Dr. Yueh’s conditioning was broken because he was more loyal to his beloved wife than to his liege.

JimFive

BF SKinner • March 11, 2010 4:10 PM

To be fair it did take a bent mentat to figure out how to break the conditioning. It wasn’t only taking a hostage. The Baron had been doing that for decades.

But I’ve always figured the Bene Geserit teaching from his wife may have conflicted with the imperial conditioning.

JonS • March 11, 2010 4:39 PM

@ Roy: “They seek “to conduct high-risk, high-payoff research” ? … If it is high-risk, it is unlikely to pay off at all. If it is sure to be high-payoff, it has to be low-risk.”

Not so. One axis is risk, other axis is reward. The four quadrants are
1) low risk / low reward (aka low hanging fruit)
2) low risk / high reward (aka the sweet spot)
3) high risk / low reward (aka only a fool would go here)
4) high risk / high reward (aka where DARPA and now IARPA play)

MBY • March 12, 2010 2:38 AM

I smell pseudoscience… Another “goat staring program”. 🙂

GreenSquirrel • March 12, 2010 3:04 AM

@Clive Robinson

The “liberal laws” have aded and abeted the rise of “youth terroristic” activities in that they know that the law has few penalties for them so shooting, stabing, rape and even murder are routien activities for gang members to get their “colours”.

To be fair, there isnt a real causal chain here.

The laws are as robust as they ever were in relation to murder, assault, rape etc. If anything we have got a lot less liberal in that the age of criminal responsibility is a lot lower.

Youth Gangs have always been violent and essentially out of control. The violence between Mods v Rockers in Brighton is the most notable and I dont recall seeing such overt gang activity today. In contrast it was almost normal in the 1960s. While there were less guns (in the hands of children, more in the hands of adults) I certainly remember it was normal for the more scum element of children at my secondary school to have bike chains, coshes, stanley knives and the like.

Despite this, and the attendent violence that was pretty much an every day occurence, less teenagers went to jail.

Anecdote Alert

I remember a fight between my school and a nearby one with in the region of 50 kids from each – the violence was pretty horrific by any standards (bottles and bike chains were the main weapon, along with odd things made in craft lessons) and I dont think any of the children (13 – 15 year olds) came away without bloody wounds. Several local windows were broken and it took the arrival of multiple police patrols to stop the main body of the fighting. Two children from my class were hospitalised with serious facial injuries but there were no deaths.

The police detained in the region of 20 of the combatants, and certainly will have known who the 20 or so others that needed to visit A&E were.

Not one person was charged with any offence. Not one child had their future ruined by a criminal record for this event. Not one child was treated as an adult despite being too young to have an adult say in his life.

I really fail to see how our laws are more liberal now.

Winter • March 12, 2010 3:39 AM

@GreenSquirrel:
“Not one person was charged with any offence. Not one child had their future ruined by a criminal record for this event. Not one child was treated as an adult despite being too young to have an adult say in his life. ”

So, by today’s reasoning, all these children must have become hardened criminals costing society millions.

Instead, if they would have been send to maximum security prison, they would all have grown up to become respectable citizens, eg, CEO at a bank.

[/sarcasm]

Winter

GreenSquirrel • March 12, 2010 4:34 AM

@ Winter

Indeed….

Prison magic 🙂

mcb • March 12, 2010 3:35 PM

@ CuriousGuru

“My cheapskate prediction is that this will fail to generate anything significant or useful.”

Well, it is generating cash flow for qualified vendors. Projekt Trustiness need never deliver a working product if the initiative’s primary goal is to keep some of IARPA’s favorite firms working. A cynic might wonder if TRUST actually stands for:

Tricks for Redistributing Under-handed Servings of Tax-dollar$

mcb • March 12, 2010 4:58 PM

@ CuriousGuru

“My cheapskate prediction is that this will fail to generate anything significant or useful.”

Tricks for Redistributing Under-handed Servings of Tax-dollar$

Bruce Clement • March 12, 2010 6:43 PM

@roy ‘They seek “to conduct high-risk, high-payoff research” ? If it is high-risk, it is unlikely to pay off at all. If it is sure to be high-payoff, it has to be low-risk.’

Ah, no. This means the research has a high risk of failure, but if it succeeds it will have a very high value.

I’m not about to try and guess what their chances of success are, but I’ve recently had a conversation with a man who invests in technology development businesses. He looks for businesses that if they work will give him better than 20 to 1 pay-offs, and the reason he looks for this is that even when he thinks the business has a good chance the reality of businesses in his field are that the vast majority of them either fail or never produce a return. His competitors, apparently, use similar logic with each picking how much risk they will take for the anticipated payoff, but each looks for the situation where (n-1)/n plans fail but the one that succeeds gives greater than n times return on investment.

Dollars don’t lie and if this research was conducted by people like this, they would decide at some point that it was failing and pull the plug. If the people running IARPA are sensible they will do the same at some point, but it isn’t their dollars. The real risk is that it will fail and for political reasons they will be afraid to admit this to their bosses so will claim a great success where there is none.

Then the fun really starts.

Pat Cahalan • March 12, 2010 7:40 PM

Dollars don’t lie and if this research
was conducted by people like this,
they would decide at some point that
it was failing and pull the plug.

Wow, boy howdy does the recent real estate bubble’s “pop” stand as a counterexample to this idea, as the Internet bubble before that, and the (fill in the bubble) bubble before that…

Government or private business or individual citizens, there’s always a bunch of dumb suckers with a lot of money who are willing to pour it down a hole with a very low chance of return… but a high payout chance… if it succeeds. “Their money” or otherwise.

This pays the electrical bill in Vegas and keeps the various state lotteries in business.

Side note: how many entrepreneurs look at your real risk as a viable payoff -> “We don’t need to make a workable product, we just have to have a good enough dog and pony show to convince the investor (in this case the government) that we’re this close to having a workable product. Repeat for profit!”

BF won the thread up at the beginning. Unless and until someone shows solid theory in basic research to support the possibility of producing a viable method of measuring psychological state via highly accurate empirical methodologies, this deserves dollar zero.

I expect any workable method to require blood chemistry analysis for hormone measurement in addition to all the other, much less accurate methods mentioned in the OP.

Charles H. Green • March 14, 2010 11:42 AM

As someone who works in the trust field, I can give you one very useful free piece of design advice. Never use the word ‘trust’ without an object.

I trust my dog with my life–but not with my ham sandwich. I trusted Clinton with the economy, but not with my daughter. (For George Bush, invert that last one).

The point is: what are you trying to measure? -Trust that they’re telling the truth? Then just stick to lie-detection.
-Trust that they’ll behave a certain way toward strangers? Read Eric Uslaner and Francis Fukuyama on propensity to trust others.
-Trust that they’ll reliably do what they say? Investigate past performance, because unlike financial securities, it is a good guide to future performance.
-Trust that they have your best interests at heart? Find a sincerity measure.

It really does all depend on what the object is of the ‘trust’ you’re looking for.

thecoldspy • March 14, 2010 4:37 PM

Trust is a very interesting subject and one in which there is plenty of study on. I looked into some of it and wrote this about it.

http://www.thecoldspy.com/the-dead-zone/trust-through-deception-thou-shalt-wage-war/

I think in the end, trust is something that allows you to become vulnerable to betrayal. Your acceptance of that parameter is when you give that trust to someone else. What happens in the end is up to those you gave that trust to. It is however a very good subject.

averros • March 15, 2010 5:17 AM

Don’t forget to pay your taxes. Your country desperately needs new creative ways to humiliate and abuse you. Enough of that free men nonsense, fall in line and prepare to be brain-searched.

mcb • March 15, 2010 9:19 AM

@ MBY

“I smell pseudoscience. Another ‘goat staring program.'”

If IARPA’s “high-risk, high-payoff research” ever results in truthiness detectors, Manchurian candidates, loyalty conditioning, mentats, or psychic assassins I guess it’s important to be first. If anyone can kill a goat just by looking at it I want that killer to be an American! 8 )

Stephanie • March 15, 2010 12:20 PM

@Petréa Mitchell great point.

I wonder if good research and checks and balances aren’t better ways to get reliable data and to discern who can be trusted? How about knowing people by their fruits? You can get a good idea of that.

Even so as Petrea pointed out, you can have an honest person/trustworthy helper who misunderstands or misinterprets. It wouldn’t mean deception, just error. For that you need a bigger perspective and an independent audit process. It seems like a lot of critical thinking skills are being overlooked by magical thinking programs.

kangaroo • March 17, 2010 9:43 AM

I love the kind of thinking that an algorithmic “lie detector” can be produced that is more successful than the product of millions of years of evolution under the most stressful conditions for the highest rewards.

My god, what a scam. Lying and lie-detection is at the heart of human evolution. You can’t beat a brain at that.