CRS Report on Polygraphs
Interesting report, especially pages 6-7 (the bit about false positives).
Comments
bmcmahon • March 21, 2007 7:03 PM
I have taken two polygraph tests in my life. The first one, I failed. I was called back for a re-test, and passed.
In one instance, I told the truth. In the other instance, I lied.
Guess which was which.
gfujimori • March 21, 2007 7:12 PM
If polygraphs don’t work so well, why hasn’t the federal government looked to the benefits that astrologers can provide in telling about someone’s integrity.
For instance, an astrologer might figure out that a new applicant is really a Scorpio on the cusp who tends to be overly loquatious and has a very bad aura, meaning they shouldn’t be given a clearance for Secret but Confidential would be ok.
gfujimori • March 21, 2007 7:22 PM
I like this quote:
“…and increasing public
confidence in national security organizations….Such utility derives from beliefs about
the procedure’s validity, which are distinct from actual validity or accuracy.???
In other words, we’re BSing the public into thinking that we actually possess the elusive oxymoron that is national security. I’m glad that we’ve got advanced technology to tell if someone really has shampoo in their carry-on.
How does this stuff register on your BS detector, Bruce?
Rob Shein • March 21, 2007 10:54 PM
Interesting…they allude (without knowing it) to the base-rate fallacy as it applies here. They talk about how the number of untrustworthy individuals working among the DoE population is extremely low (well under 1 in 1000) as compared to the rate of “positives” resulting from polygraph testing. Good to see simple logic being used, along with elementary-level statistics.
Gino • March 21, 2007 11:28 PM
President Nixon said, “I don’t know if they work, but I know they scare the hell out of people.”
merkelcellcancer • March 22, 2007 12:03 AM
Just a footnote to this article, In its 2002 report, NAS concluded that polygraphs as currently used to screen applicants have serious limitations, and that the accuracy of the polygraph in distinguishing actual or potential security violators from innocent test takers is insufficient to justify reliance on its use in employee security
screening in federal agencies.
World’s best lie detector: mom.
Detector Clouseau • March 22, 2007 3:00 AM
BS not only knows his position and momentum but needs no detector.
geomark • March 22, 2007 6:10 AM
I had to take two polygraphs for a certain agency that goes by a three letter acronym. First time I told the truth, but they said I was being deceptive about my use of recreational drugs as a youth. That was a long time ago and pretty small time stuff but I had told them the full extent of it.
So I came back for a second test and this time I expanded on everything, telling them what they seemed to want to hear. I told them I had tried anything and everything I could get my hands on and it was great. They said I was being deceptive.
Gomez • March 22, 2007 6:24 AM
A little light went on when I saw this was a Department of ENERGY report. Check out their history — started after the 1973 energy crisis and now they do nuclear weapons and intelligence operations.
You learn something new every day…
George Maschke • March 22, 2007 8:00 AM
I’m a co-founder of AntiPolygraph.org, which was linked above by Sceptic. For a critical overview of polygraph validity, policy, procedure, and countermeasures (how to pass or beat the polygraph), see our e-book The Lie Behind the Lie Detector (1mb PDF):
http://antipolygraph.org/lie-behind-the-lie-detector.pdf
Our government’s misplaced reliance on the unreliable pseudoscience of polygraphy undermines national security and public safety. We needn’t wait until a working lie detector is invented before abolishing one that is a fraud. AntiPolygraph.org has proposed language for a Comprehensive Employee Polygraph Protection Act that would close the governmental and other loopholes in the existing law:
Anonymous • March 22, 2007 8:52 AM
http://www.polygraphacademy.com/manual.htm
So what’s the method used by sex criminals to beat the polygraph?
Corey • March 22, 2007 8:54 AM
Polygraph control questions have always kind of worried me, though I’ve yet to have to do anything requiring such a test.
From what I understand, some of the questions in a polygraph you’re expected to lie to, but they of course don’t tell you which. I’m pathologically honest (“yes, honey, that outfit isn’t very flattering, you’d look better in another”). Example: “Do you ever think about cheating on your wife?” “Of course!” “Have you ever actually cheated?” “No”
Having the “lie” baseline be essentially the same as the “truth” baseline (which is what would happen if you answered them all truthfully) would result in lots of false indications of falsehood.
Bryan Feir • March 22, 2007 9:30 AM
@Rob Shein
I don’t know about the ‘without knowing it’ part. I suspect the person who wrote that is familiar with the base rate fallacy, it’s just that his intended audience is politicians who have no familiarity with the fallacy in question but do have an unwarranted belief in the capability of the polygraph. In that sort of situation, you use simple descriptions.
IMHO, it is also important to note that the polygraph is, nomen est omen, just a device that graphs many (poly) pieces of data. This data is then interpreted by the operator. You are at the mercy of the operator, who can make or break you. It’s a situation where the beggar is before the king, bowing to its absolute power. If that doesn’t make you nervous and sweaty, I don’t know what will.
The operator’s authority is unchecked, because he/she can simply claim that the evidence from the polygraph — just a jumble of lines on a piece of paper — is irrefutable.
I think a polygraph is a pseudo-science equivalent of a psychological evaluation, and we all know how flawed they can be (repressed memories, anyone?).
Also, a polygraph exam is based on a presumption of guilt: you have to prove (by way of your supposedly involuntary reactions) that you have nothing to hide.
Re: “So what’s the method used by sex criminals to beat the polygraph?”
Just sit back and think about baseball?
Alex • March 22, 2007 6:53 PM
The polygraph is nothing more than a modern day version of Trial by Ordeal (http://en.wikipedia.org/wiki/Ordeal).
Star 330 • March 23, 2007 3:42 AM
My father was a polygraph examiner for a large U.S city’s police department and during his “set up” he would have the examinee choose a card then he would go through each “was it the ace of spades?” and so forth then tell them which card they choose. Of course it was a trick but they would be left thinking it worked. If you think it works it most likely will. If you think it doesen’t it most likely won’t.
Carlo Graziani • March 25, 2007 11:49 AM
For those who have not perused the 2002 NAS report, I highly recommend it. Not the whole thing, necessarily, but just skimming it is a joy. Chapter 2 (“Validity and Its Measurement”, pp. 29-64) is a gem. It is a treasure trove of material on quantitative assessment of screening tools and tests. It defines the terms “Reliability”, “Accuracy”, and “Validity” in clear and precise language, and even supplies a lucid illustration of their meaning in the context of airport X-ray luggage screening.
The section I found of most value is “A CONSISTENT APPROACH TO MEASURING ACCURACY” (pp. 37-51). In particular, the figures on pp. 44-45 showing the relation of the receiver operating characteristic (ROC) curve to choice of sensitivity threshold is the perfect model of how one should think of false positives/false negatives, and their influence on testing.
These plots should be compared to the empirical data on polygraph testing, and particularly to the plot in Chapter 5, on p. 123. The typical false-positive rate necessary to generate a sensitivity (== “true-positive rate”) of 0.65 (65% chance of catching a real bad guy) is about 10% (one in ten good guys fail test). Higher sensitivity rates cost (much) higher false-positive rates.
Note that a sensitivity of 0.5 is essentially a coin-flip test. So if you’re thinking “a sensitivity of 0.65 isn’t so bad”, think again. It’s abominably bad, really only barely better than blind guessing.
What these plots show is that government screening polygraphs do not possess a high-sensitivity, low false-positive regime. Depending on the sensitivity settings, they range in value from coin-flip to “Straight J’Accuse”. There are no scientific grounds to distinguish them from ouija boards.
Subscribe to comments on this entry
Leave a comment
Sidebar photo of Bruce Schneier by Joe MacInnis.
Roy • March 21, 2007 5:41 PM
There are so many things wrong with polygraph use that I don’t know where to begin.