Schneier on Security
A blog covering security and security technology.
« Friday Squid Blogging: Squid Beer Ad |
| An Expectation of Online Privacy »
May 4, 2009
This may be the stupidest example of risk assessment I've ever seen. It's a video clip from a recent Daily Show, about he dangers of the Large Hadron Collider. The segment starts off slow, but then there's an exchange with high school science teacher Walter L. Wagner, who insists the device has a 50-50 chance of destroying the world:
"If you have something that can happen, and something that won't necessarily happen, it's going to either happen or it's going to not happen, and so the best guess is 1 in 2."
"I'm not sure that's how probability works, Walter."
This is followed by clips of news shows taking the guy seriously.
In related news, almost four-fifths of Americans don't know that a trillion is a million million, and most think it's less than that. Is it any wonder why we're having so much trouble with national budget debates?
Posted on May 4, 2009 at 6:19 AM
• 138 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
"If you and me are going to be the last two people on earth, Walter, we ... we should try breeding."
"I don't think that's going to work."
"Well, it's like you say, it's 50-50. There's a 50% chance it'll work."
"There's a zero percent chance it'll work."
How could this guy be wrong? He's a high school science teacher after all. Nobody would allow an ignoramus to teach their children, would they?
@SimonC: You beat me to it. That was the best part. I doubt if Walter figured it out though.
Well, that particular fallacy is not unheard-of even among people who ARE mathematically literate otherwise. Remember the Monty Hall Problem, and how many smart people initially insisted that would be 50-50?
The difference between 1/3 and 1/googol is only one of degree, not one of logic.
That guy is a *teacher*? Even more so, a *science* teacher? What's next - Gene Ray as the head of the National Science Foundation?
This is new heights for the Daily Show - Bruce Schneier is using it to illustrate security points!
Here's another person who thinks the same way:
"In the absence of scientific evidence, there is an equal chance that any therapy will be beneficial or harmful."
The results of the "trillion" survey are frightening! It's a multiple-choice question with six possible answers (one of which is "don't know"), so choosing *randomly* should lead to about 17% accuracy. The actual figure is 21%, which means people are doing only slightly better than chance...
It is good news that 79% of Americans reject the false teaching that a trillion is 10^12. However, the truth (which is that a trillion is 10^18, i.e. a billion millions) is still being repressed; it was not even among the options.
To complicate things: (at least) in Germany a "Trillion" is a million million million (10^18) - the counting goes:
10^3 Tausend (thousand)
10^6 Millionen (million)
10^9 Milliarden (billion)
10^12 Billionen (trillion)
10^15 Billiarden (quadrillion)
10^18 Trillionen (quintillion)
It always is quite unsettling whenever a translator mis-translates a US deficit/spending in trillions as "Trillionen" (quintillions)...
The exchange was very reminiscent of a recent QI episode (a British comedy panel show). Episode 6x06, at 25:50. Aired 30-Jan-09.
Jimmy Carr: "I'm with Alan. Everything's 50/50 isn't it. Winning the lottery - 50/50. You either win it, or you don't. Rolling a six - you either roll a six or you don't. 50/50."
I cannot fathom that this science teacher doesn't understand probability. It manifestly must be a joke.
Scientific notation ftw.
million = 10^6
US trillion/European billion = 10^12 = 10^6 * 10^6
Funny, the computer screen the "teacher" is looking at is the windows screen showing after a system crash.
The fact that there is a science teacher somewhere who doesn't have a clue is not surprising. The problem is that the cable news media also can't think and gives this guy air time. It seems the 50-50 idea also applies to editorial decisions in the media, if there are two sides to an issue, they are both given equal attention even if one lacks any credibility.
Around here a trillion is 10^(3*6) = 10^18, I like the system better than the american one. But I suppose it's just too systematic.
>> "In the absence of scientific evidence, there is an equal chance that any therapy will be beneficial or harmful."
In the absence of evidence, you can't base your a priori probability on anything, so 50-50 is as good a number as any; maybe even the better one because it is unbiased toward either result. Of course there are few, if any, phenomena where there is a complete absence of evidence for and against.
And there is also the difference between the actual probability of something happening, and the best guess you can make for it. Suppose I flip a coin and keep it hidden, and then you pick heads or tails; I will know with 100% certainty whether you've got it right or wrong, but you'd still be left with 50-50 odds, unable to decide whether to switch or not.
A probability of '1 in 1,000,000,000' does not mean you have 999,999,999 instances before you get struck by lightning, run over by a buffalo herd, etc.
People tend to neglect the importance of statistically independent events. That is, if I'm playing craps and haven't rolled a seven in my last twenty rolls, the odds of rolling a seven on my twenty-first roll are different than rolling a seven on any given roll of the dice.
++Nick. I've seen a lot of very bright people, medical doctors even, who did not understand such basic statistical concepts as independent events. (Dice don't have memory!)
As for teachers: Back in grade school I told my teacher that a million times zero was zero. She didn't believe me. She thought my answer was too low.
Consider for a moment who tolerates the low pay and overwhelmingly politics of a career in teaching. Whom does it attract? It's like considering who is attracted to a low-paying high-stress job of standing in front of an airport X-Ray machine for 8 hours a day. And then we wonder why these people behave the way they do.
The point about "trillion" is not whether the U.S. conventional usage is better or worse than those of the UK, Germany, or others.
The point is that in the U.S., there is a conventional usage, and it is used frequently in important policy debates about fiscal and budgetary matters. A citizen who doesn't know what a "trillion" is can't access those debates, because he can't relate any numbers to the size of the US GDP or to that of the US Federal budget.
The fact that 80% of US citizens don't know how to use "trillion" is in its way more dispiriting than the poll numbers on creation/evolution. The aforementioned debates might as well be conducted in Latin, as far as they are concerned.
@George: "The fact that there is a science teacher somewhere who doesn't have a clue is not surprising. "
No it isn't. The probability is obviously 50-50, so be glad that it doesn't happen more often... ;-)
Bruce's comment on people's understanding of trillion reminds me of this XKCD comic:
"if I'm playing craps and haven't rolled a seven in my last twenty rolls, the odds of rolling a seven on my twenty-first roll are different than rolling a seven on any given roll of the dice."
No, it's not! Your chance of rolling x number of times without rolling a seven gets smaller as x gets bigger, but your chance of rolling a 7 on any given toss still stays the same because they're, you know, statistically INDEPENDENT events.
>That is, if I'm playing craps and haven't rolled a seven in my last twenty rolls, the odds of
>rolling a seven on my twenty-first roll are different than rolling a seven on any given roll
>of the dice.
Er, did you just "neglect the importance of statistically independent events". The odds of rolling a seven after twenty rolls are exactly the same as after the first roll.
But you knew that, right?
OK, everyone. Let's break out our slide rules and remember to add the exponents:
10^12 = 10^(6 + 6)
The saddest part of this is that Walter Wagner has advanced degrees. He's referred to as Dr., although I think that comes from a (JD) law degree.
Nice, but harsh, Wagner background check:
Before we come down on the trillion people, maybe we should ask why they might have made the mistake *other* than being bad with numbers. To me, it seems like the mistake is in the range/orders of magnitude. That is, people might simply have thought 100,000,000 was in the millions so, yes, trillions were only 1,000 times away. I think that people were probably confused by where million stops and trillion begins, possibly making it more a mistake of the question's English than the answer's Math.
To put it another way, I could probably show that even most Computer Science grads are idiots by asking them "A character can be represented by a single 8-bit value; how many characters can you represent with a second 32-bit value?" Depending on how you parse the question, it has 4 "correct" answers. Sometimes it's the questions that are wrong.
Say I flip a coin a trillion times and it always comes up heads.
The probability of the next coin toss being tails is still 1/2...right?
After 1 trillion and 1 trials, a distribution of 1 trillion heads and 1 tails is pretty small...right (assuming independent events)?
And, finally, the reason there is a difference is because we are discussing two different propositions that we are trying to test...right?
One of them is about the results of a single coin flip and one of them is about the result of 1 trillion and 1 coin flips. The confusion comes in because people don't realize that they are asking different questions of different data sets--they only see the thing happening, which is that I'm flipping a coin.
I was recently told by a stats grad student "That's not how it works."
Correction to my post: both are 8-bit values! As if the question weren't bad enough . . .
A science teacher who doesn't know probabilities? Next you'll be telling me Al Gore doesn't know global warming! ;)
"That is, if I'm playing craps and haven't rolled a seven in my last twenty rolls, the odds of rolling a seven on my twenty-first roll are [no] different than rolling a seven on any given roll of the dice."
However, it's pretty unlikely not to have thrown any sevens after 20 times, so I'd start suspecting foul play. You _should_ adjust your expectations of your next throw based on the history you have with the dice. The probabilities of your dice don't change, but your information about them does.
Of course the reality is that people often change their expectation in the opposite direction. They think "I haven't thrown a seven in ages, I'm about due a seven", rather than "I haven't thrown a seven in ages, these dice are loaded and I'll probably never throw a seven"
See http://digg.com/general_sciences/... which discusses and links to other discussions of Mr. Wagner's lack of credentials. Unless shown some proof I doubt he's a teacher anywhere.
I thought it was a thousand billion.
Not to defend the guy, but he did say "best guess", when in actuality, with no data any guess is a best guess, right? Although he was saying that it automatically implies 50/50. The thing is that 50/50 probability only has a 1:2 chance of being correct itself, ad infinitum. ;-)
Furthermore, the scientists that are so sure their math shows that the end of the universe being caused by their experiment, though not impossible, is extremely unlikely, failed to calculate the probability that they messed up the first time and it would explode when they turned it on, which, in hindsight, was 100%.
We should be offering more salaries to teachers and more pension and other benefits - then only we can find really good educators.
Is this guy a plant by NEA to show how bad the teachers are and who much better they will be if spent another Billion/Trillion (who cares) on higher pay for teachers?
Education is a farce because it's a "right"
To the people criticizing my original comment about statistically independent events, perhaps I wasn't clear:
Odds of rolling 7 on 21st roll - one event.
Odds of rolling 7 on any roll - different event.
People tend to confuse the two, believing that the odds on any given roll change because of prior instances. This leads to people picking 'lucky numbers' on lottery tickets, or doubling their bet when they lose a hand at blackjack.
nonny bunny: However, it's pretty unlikely not to have thrown any sevens after 20 times, so I'd start suspecting foul play.
And we have a winner! In real life games, you can't assume that the rules are consistent -- i.e., that the house isn't cheating. Which is one of the underlying problems in any "risk assessment" -- they're mathematical games, unlike real life games.
Yeah, most people are stupid -- including people who are ostensibly smart. Risk assessment only gives you heuristics that are useful if not taken to their logical conclusion. They ain't "true".
Just a simple technical note. When you don't know anything about a distribution of outcomes, the middle is the best guess you can make.
Well, when someone says something that sounds amusingly wrong, it's often a communication problem. Usually, assumptions are made and not explicitly stated. In this case, I think the unstated assumption is that there is no knowledge of the situation - therefore no basis whatsoever for an assessment of probability. In that case, 50-50 is a logical estimate of the odds.
HOWEVER, it's completely useless for any practical purposes! Since it is based on absolute ignorance, it can't possibly help us to decide how to proceed. Therefore, the only thing it does tell us is that we need to learn more - urgently, if the decision is urgent.
Mr. Wagner is either a science teacher, or not a science teacher: therefore the probability that he works as a science teacher is 0.5 .
A science teacher either understands probability, or doesn't: therefore the probability that any particular science teacher understands probability is 0.5 .
Accordingly, the probability that Mr. Wagner is a science teacher who understands probability is 0.25 ... or anyway, it's probably 0.25 .
On the coin toss issue, the disagreement arises because people are conditioning on different information.
Conditioned on the presumption that the coin is fair (or at least that the heads probability is known), the trillionth coin flip is independent on all the previous flips, and the probability of heads is unchanged. Even if you had 999,999,999,999 heads in previous flips, you're not conditioning on that information, so it doesn't matter.
On the other hand, if you're using the previous N attempts to _estimate_ the unknown heads probability, and use that estimate to calculate your chances of getting heads at the N+1 attempt, it is obvious that the probabilty will change as a function of the previous attempts.
If I recall correctly, Laplace first did something like this with "the probability the Sun will come up tomorrow". All you need is a relatively uninformative prior probability, which you update using the data to give you a posterior probability. With even only a little data, the specific form of the (admittedly subjective) prior ceases to matter, because the posterior becomes dominated by the probability of the data (the likelihood function).
Wagner's problem (well, his statistical problem, anyway) is that he has no data -- the stringy BS invoked to make the LHC blow up the world is so unconstrained by data as to border on theology -- so his posterior _is_ his prior, which is utterly, irredemaby, subjective, akin to a statement of faith.
Over 3 in 2 people in the world are completely innumerate. This is the opposite of bubblegum.
"...John Allen Paulos published the book Innumeracy. In it he includes the notion of chance as well to that of numbers."
Does anybody has the good teacher's contact information? I have a couple of even-money bets for him...
Nick: the "doubling your bet" phenomenon is completely different -- it's even got a mathematical analysis in which it's sound, although the assumptions needed to make it work in practice are unrealistic.
(Suppose that whenever I lose, I double my bet for the next iteration. I start betting $1, if I lose, then I bet $2, if I lose again, then I bet $4, et cetera. Whenever I win, I start the next bet at $1 again. It's not too hard to see that between two wins with n losses between them, my cumulative loss has been 1 + 2 + ... + 2^(n-1), which is (2^n) - 1. On the other hand, my bet is 2^n : so, if the game pays off even, the stretch between two wins always results in me gaining a dollar. So, I've proven that as long as I leave the game immediately after any N wins, I will have made a net profit of $N. QED. Of course, if you try this in practice, you run into the problem that, if your wallet can only stand losing, say, a million dollars, then any particular run where you go for more than 20 losses will immediately bankrupt you, and you can't keep trying more until you win. A 50/50 chance of winning gives a 1/2^20 chance of getting bankrupted between wins, meaning that you get on average 2^20 wins -- and 2^20 "permanent" dollars -- before you get bankrupted by the unlikely event. That probability comes back to kick you in the arse this way should not surprise anybody. =D )
"almost four-fifths of Americans don't know that a trillion is a million million"
This is why no-one seems concerned about a 3 trillion dollar federal budget but the same people get up in arms about 125 million dollars in bonuses. Certainly 125 is larger than 3!
I remember hearing the same "logic" come out of the mouth of an engineer I worked with several years ago, nearly word-for-word. It's even more disturbing hearing it come from an educator, since you know the faulty thinking is being passed on to others. I love the way the guy chooses when to apply this "reasoning" and when to abandon it (at the end of the video).
Maybe I'm being a little dense, but which part of Mr Wagner's statements is/are the worst offenses against mathematical literacy?
It seems reasonable that in the absence of any other information, the optimal estimation that the probability that an event will occur is 50-50.
In that case, it seems that the stupidity is in the fact that you're trying to make an educated estimation while claiming to have no reliable information?
Is the offense that people are believing that this is an educated estimation when it's not?
Tim, I'd be miffed if I learned that anyone involved with the banking fiasco was getting a $0.57 bonus. (Or even a $0.0057 bonus for you XKCD fans.)
A trillion dollars for things like the navy and the interstate highway system don't seem like that bad of a deal. $125000000 bonus for adding another $1000000000000 to the national debt, on the other hand....
The good news is that nobody really believes this guy. He doesn't even believe himself.
How can I know? Easy.
Take a random person off the street. Tell him you're going to flip a coin. Heads, he walks away. Tails, you shoot him in the head. What happens?
It's virtually certain that he will act as though you're trying to kill him, and rightfully so. He'll go nuts, attack you, try to run away, start crying, something like that.
This situation is equivalent, and yet nobody does anything beyond talking on the news or filing lawsuits.
If anybody truly believed that the LHC was a coin-flip away from killing them, they'd be marching on the site with torches and pitchforks, not appearing as a guest on an American comedy show.
If I haven't rolled a 7 in a while maybe I should try rolling two dice.
peoples! you're taking this way too seriously. it was obviously a farce!!!
@Nick Lancaster: Your initial comment is poorly worded (or maybe it contains an accidental omission).
If you had said "the odds of rolling a seven on my twenty-first roll are no different than rolling a seven on any given roll of the dice." (Notice that I stuck a "no" in before the different there), no one would be complaining, but the way it is worded, it is easy to interpret it as if those first 20 rolls change the next one.
"Ominously rounded hallways..." lmao
Apparently Canada and the US have a different definitions of a trillion (10^12 vs 10^18) (except for in Quebec), So I can understand where the confusion comes in. And I will now be significantly more horrified at the size of the US debt.
One thing in math/statistical literacy questions that has been addressed here is that the question has to be understood first. When you want to talk purely of random chance, you never say "when you flip a coin", you say "when you flip a fair coin" so there is no misunderstanding. Lack of information can bias the answer.
That science teacher probably also says "less people."
To those pointing out the difference between the North American trillion (10^12) and the European trillion (10^18):
Note that the European trillion wasn't one of the responses in the poll at all, and that most people thought it was less than 10^12. After a million million (10^12) at 21%, the next most common answers were:
- a hundred thousand million (10^11) at 21%,
- a thousand million (10^9) at 18%, and
- ten million million (10^13) at 17%.
Honestly, the main thing I get from this poll isn't so much that people have no clue about big numbers (which is also true) but that most people are unwilling to admit that they don't know the answer to a question.
The correct answer is of course that a trillion is larger than a million every time. It doesn't suddenly stop being larger after having been larger one million times.
"Honestly, the main thing I get from this poll ... that most people are unwilling to admit that they don't know the answer to a question."
I already knew that. No, really, I did. I swear.
I guess this explains the expansion in the number of American casinos, state lotteries, etc.
@"unknown" == "50/50" folks:
> In the absence of evidence, you can't base your a priori probability on
> anything, so 50-50 is as good a number as any;
Not really - a logical Truth Table gives possibilities, which doesn't translate to probabilities. "Two theoretical possibilities whose probability are unknown" is not the same thing as "two possibilities of equal probability."
Since extraordinary claims require extraordinary evidence, a better guesstimate of probabilities would be to divide the strength of the evidence by the severity of the claim.
For example, a black hole consuming our planet has severity approaching infinity, and particle physicists say the evidence pointing to it happening is about zero. About zero divided by about infinity gives us roughly a zero chance it'll happen.
Or, in the hypothetical where the evidence (or lack thereof) favors neither possibility, the best approximation would be to go with the possibility of least outrageous impact.
Maybe I'm reading this wrong. That logo in the bottom left reads "Comedy Central", right? Why the heck is everybody discussing what is most likely just a satirical view of exactly the ignorance you are discussing?
Hey, I'm from South Africa, and don't watch a lot of TV, but from what I can recall Comedy Central isn't exactly CNN, so why so serious?
Interesting term I can now use to describe things: "statistically independent events". Know the phenomenon, but now I've got a name for it.
The Daily Show parodies normal news programs in its presentation, not in its facts. Guests are real and sometimes their reactions are real.
I've always thought that computer terminology is much better at helping people grok huge numbers.
We should be talking about kilodollars, megadollars, gigadollars, and teradollars. It makes more sense, and if we all start using it, perhaps the impact of our financial decisions will be understood more clearly.
What's worse is that I've heard PHDs publicly make way more completely incorrect and dangerous claims than this science teacher.
> John Stewart (and to a lesser extent, Stephen Colbert) are probably the most trusted names in US news precisely because they don't take it as seriously as the mainstream news outlets of CNN, MSNBC, or FoxNews.
I don't think its that the mainstream news outlets take US news too seriously, it's that they take themselves too seriously.
Case in point, the flack that Sara Palin took, all arguments about her qualifications (or lack thereof) aside, because she didn't know what "The Bush Doctrine"(TM) was, was really the news media upset because she didn't follow them as closely as they insist people follow them. "The Bush Doctrine"(TM) was/is a term wholly made up by the media themselves and is/was frequently redefined by the various news outlets. When you villanize the person you're interviewing because they don't use the same vocabulary as you (especially when it's vocabulary created by you) that's just lazy reporting (par for the course I guess).
> They have no problem asking the questions that a regular person actually wants answers to. They do not pull punches to keep someone from looking bad.
This is the genious of those shows (and of political satirists in general). They are freed from the shackles of political correctness and thus can get to the point more directly.
However, it kills me to see some of the avid supporters of political satirists who insist that their satirist is objective in their reporting or that by only listening to them, they somehow understand the complexities behind the issues reported on the show. Both John Stewart and Stephen Colbert have very obvious left-leaning spins in their shows. That's fine (everyone has their bias), but don't forget that their goal is give their audience what they want (which generally isn't in-depth coverage of different sides to the issue).
@ Sarah Davies
Those are SI prefixes and are not specific to computers. E.g., kilometers, centimeters, kiloliters, kilograms, giganewtons, megapascals. You might recognize the first one from Canada, the second from a ruler, and the rest from science class. ;)
I do agree with you entirely.
50/50 is arguably the best guess in the absence of *any* evidence (a very very rare situation). If the probability can be 0-100 with equal probability, 50 has the lowest expectation of error, for instance.
It's probably more useful as a bound. For instance, if something hasn't happened in x years, assume it doesn't happen more than every 2x years, (if it happens more than every 2x years, the odds of it happening in any given x years is greater than 50%.
"I've always thought that computer terminology is much better at helping people grok huge numbers."
I'm guessing you work mainly with people who know a lot about computers. For the average non-computer person, gigadollars and teradollars won't be any more meaningful than billions and trillions. Less, because they're new and unfamiliar terms. (Though it would at least do something about the transatlantic confusion.)
The human brain just isn't equipped to deal with these sorts of numbers instinctively. That's why textbooks and news stories are forever going to the silly-sounding analogies involving skyscrapers or football fields (whoops, more transatlantic confusion).
Trouble is, nobody has an audience which wants in-depth coverage of different sides, or even accuracy in general.
I would never pretend that The Daily Show is good news, but sadly it is the *best* news currently available on television.
I'm surprised nobody has brought up Schrödinger's cat. Inside a sealed box, the cat is either dead or it's alive, but we won't know for certain until we open the box. Is it simultaneously alive and dead? No, of course not; we just don't know what state it's in until we open the box. And until that time, the odds are 50/50, aren't they?
So when the science teacher said, "If you have something that can happen, and something that won't necessarily happen, it's going to either happen or it's going to not happen, and so the best guess is 1 in 2", what he meant is either the cat is dead (as in the earth is destroyed), or it's alive (as in nothing bad happens).
He tried to simplify it for the poor comedian. Perhaps he simplified it too much for the rest of us.
WRT to billions and trillions: Like most here, I imagine, I knew what a billion was and what a trillion was in grade school. What I did not know was what a billion dollars would buy, what a trillion dollars would buy, or what kind of government would have a budget on that order of magnitude. What's generally lacking is not just the ability to write down those numbers and the difference between them, but a visceral feel for what they represent--one that matches the visceral feel we have for what a sawbuck, a C-note, or a grand will buy. (For non-US residents, those are $10, $100, and $1,000.)
@Dave: The best "guess" in the instance that you have no information is "I don't know." It is not "50-50." That suggests that those numbers have at least one digit of significance, when in fact they have none. If you can establish that the two possibilities are symmetric, you may venture a guess of "50-50," but then that is some information, not no information.
@Michael: The best news available on TV is all of it. Then use your judgment. Unfortunately, this takes a lot of time. But it is at least generally applicable.
@Glen: I don't think that's a valid argument, because in both cases, we're not operating in the absence of any information about the system. If, for the sake of argument, you assume the cat is in a superposition of dead and alive states, it does not follow that the two states are equiprobable. The longer you wait, the more likely a decay has been detected and the cyanide capsule opened, and therefore the more likely the cat is dead. Presuming you know the experimental set-up, you can calculate the probabilities fairly accurately. If you don't know the set-up, then it's pointless to guess.
Likewise, the LHC is not a black box. Even if it were, there's no good justification to just guess 50-50. But it isn't; we know a lot of the science behind it, and the threat being considered is a theoretical one, meaning there is some theory behind it. That theory, if it is any good, should allow us to say something more meaningful about the odds of something bad happening than an empty guess of 50-50.
Sorry about the triangle (triple post offense).
The probability is not 0.5 It depends on how long the cat has been in the box. The longer it is in the box, the more likely that a radioactive decay has occurred and the poison has been released. So the probability that the cat is alive is decreasing over time. The only time is 0.5 is at the time corresponding to the half-life of the radioactive material.
> 50/50 is arguably the best guess in the absence of *any* evidence
If you were arguing with a statistician, you'd lose that argument. You're conflating probability with possibility.
An arbitrary posit that "x might happen" does not imply anything about the probability of x. Absent any statistics, there is absolutely no basis to suppose "50-50" is more accurate than "3 out of 1.2709 * 10^39".
If you're going to be taking a lot of time then you may as well skip the TV altogether. The information/time ratio is much better in other media.
The trouble being that the average person (read: average voter) does not do this, and thus our national policies are informed by CNN and (shudder) FOX News.
"Is it any wonder why we're having so much trouble with national budget debates?"
Keeping people mathematically illiterate is a politician's wet dream.
Actually we have a ton of evidence about whether the LHC will destroy the world. It's is creating a particle collision with a certain energy. Cosmic rays with more energy hit the Earth regularly, and it has not yet been destroyed.
It's new that we're creating such events in the middle of an amazing array of sensors, but the events themselves are not at all rare.
As far as billions, trillions, and Government spending, here's a hand guide:
* $1 Billion = $10 each family will be taxed, on average (more with interest).
* $1 Trillion = $10,000 each family will be taxed.
* $12 Trillion = Ouch!
Debating "should the government spend 4 trillion" is very abstract.
Debating "should the government put each family $40,000 into new debt" is very concrete. People can relate to that sum, and would hopefully expect something *really* good to happen if the government spent that much.
Or as GWB was taught "a miyon miyon is a biyon"
On the other hand, a guess of 50-50 gives the highest amount of entropy, which would make sense given that you have absolutely 0 information about x.
"...his posterior _is_ his prior..."
Which is, incidentally, also what he's doing his best impression of thinking with.
"In related news, almost four-fifths of Americans don't know that a trillion is a million million, and most think it's less than that. Is it any wonder why we're having so much trouble with national budget debates?"
I have an issue with the fact that they conducted this over a telephone survey. What was the duration of the average call, compared to the duration of the call for the individuals who got the question right (or closer to right)? How many hung up the phone afterward and realized their mistake? How many were interrupted at dinner time (or were otherwise distracted) and gave an answer randomly just to get the pollster off of the phone?
I deeply mistrust results garnered from telephone polls, even when they confirm my natural proclivity to assume that most people are idiots.
Not so! You should condition your probabilities on the fact that you don't know anything about X, as well as on other things, like the fact that you exist at all.
For fun with the Anthropic Principle, see this lecture. (It oddly enough is actually titled "Fun With The Anthropic Principle," which I didn't discover until after I wrote that sentence and googled it.)
Game theory scenario:
You and a friend are taken into custody and put into two separate jail cells where you cannot communicate with each other. You are told some unknown experiment is taking place and that there are two possible outcomes, A and B.
You are then forced to guess the probability of A occurring. Your friend is in the same situation and you must both guess the same value otherwise you will be killed.
With no other information you would guess 50%. It is the only logical equilibrium.
Once you become an acknowledged expert you can muse about virtually anything and *people* will quiver with awe at the sheer brilliance of your observation. I mean servile flatterers.
@Volker: 'It always is quite unsettling whenever a translator mis-translates a US deficit/spending in trillions as "Trillionen" (quintillions)...'
Give us time.
According to the Oxford Concise English Dictionary, a trillion is 10^12 unless it refers to something that is 'British dated', in which case it is 10^18. Sadly, I do not have access to the full Oxford English Dictionary which, I assume, would give a more complete and meaningful definition.
As to people in the UK grasping the difference between 'point in time' and 'predictive cumulative' probabilities, or the value a trillion represents, I'd hazard a guess at
We can call anything whatever we want, if all we need is hot air.
If we want useful probabilities, we need to work from observed facts and accepted science, rather than arbitrarily declaring all imaginable possibilities equally likely.
I'll confess I don't understand the "high entropy" thing, but if entropy is what we want, probably we really could use hot air.
@i really, really need a life
The units of measurement used in computer science are NOT SI units. They all derive from the binary system, so be aware that kilo = 1024, not 1000. A kilobyte of RAM/disk is 1024 bytes, a megabyte is 1024 * 1024 bytes, and so on.
> According to the Oxford Concise English Dictionary, a trillion is 10^12 unless it refers to something that is 'British dated', in which case it is 10^18. Sadly, I do not have access to the full Oxford English Dictionary which, I assume, would give a more complete and meaningful definition.
According to the Shorter Oxford, 6th edition:
Trillion - Originally (especially in the UK), a million million million (10^18). Now usually (originally US), a million million (10^12; cf. billion)
Billion - 1. A million million, 10^12. Cf. trillion. (Now only in British popular use.)
2. A thousand million, 10^9.
What's with those right-wing websites, they crash my open source browser, on linux! Nothing else does...
On the "trillion" debate going on here, the progression is as follows:
- Mono-illion (M'illion)
- Bi-illion (B'illion)
- Tri-illion (Tr'illion)
- Quad-illion (Quadrillion)
etc etc etc. In Britain, a thousand thousand thousands is called a Million (i.e. a thousand millions), and I think a thousand thousands is also called a million-- hence why POSIX.1 establishes a million as a "Thousand Thousands" in some definitions of time functions.
My Social Studies teacher in high school argued with me, trying to prove that 800% of $1 would be $800.
The fact that common people don't comprehend a trillion is not as much of a concern as the need for them to understand the power of compounding interest and tie that in with the fact that the us dollar is borrowed by the government from the fed and there is interest to pay off on each and every dollar. Hence, money is debt, and that debt is growing exponentially.
We (were) talking about dollars which are counted in powers of ten. Anyways, some people prefer kibi-, mebi-, gibi-, etc. as the binary alternatives to the SI prefixes.
> I would never pretend that The Daily Show is good news, but sadly it is the *best* news currently available on television.
Which is why I stopped getting my news from TV years ago. I get my news now from two sources: NPR and via a specific simple routine of gathering information about a story from the internet that I've developed.
I like NPR because they have nothing but their voice to communicate with you. No flashy graphics or videos. No body language. Just words (and inflections/tones woven inbetween the words). It is far less entertaining than TV news but that's the beauty of it. They aren't there to entertain, they are there to report, and it is some of the best reporting left in the US.
As for the routine I use to get information on a story on the internet here it is:
1) Go to CNN (it's order here is random) -- this more-or-less what the left is saying about the matter
2) Go to Fox and find the same story -- this is what the right is saying
3) Go to an online guide about how to detect bias in reporting (there's lots of them out there)
4) Realize that both CNN and Fox have sub-par reporting
5) Go to news.google.com and search for that story
6) Read several articles/blogs comments on that story
7) If still not satisfied, search Wikipedia on related topics (ie: medicine, science, politics, cultures, religions, etc.)... bonus points if you end up reading peer-reviewed papers on the topic (citations in Wikipedia are wonderful)
This takes more time, but you'd be suprised how much more time you have to do this when you're not watching all the commercials and pointless/repetitive reporting on TV.
If you think math illiteracy is bad, take a look at ignorance of history, ignorance of economics and politics, and geography and foreign cultures, ignorance of basic health, psychology and the like. I know the math is a disaster but these other things have even worse consequences on *us all.* We don't need them to be engineers, until first they are competent to live and take care of themselves !!
BRIDGEKEEPER: Stop! Who would cross the Bridge of International Depression must answer me these questions three, ere the other side he see.
What... is your name?
OBAMA: It is 'Obama', President of the United States.
BRIDGEKEEPER: What... is your quest?
OBAMA: To fight the Global Recession.
BRIDGEKEEPER: How... many millions are in a trillion?
OBAMA: What do you mean? An American or European trillion?
BRIDGEKEEPER: Huh? I-- I don't know that. *Auuuuuuuugh!*
BIDEN: How do know so much about trillions?
OBAMA: Well, you have to know these things when you're a President, you know.
You, sir, win the Internet!
> With no other information you would guess 50%.
> It is the only logical equilibrium.
No, there *is* no logical equilibrium. You have *no* data.
With the scenario you are given, you don't even know what the significant figures are.
Without anything resembling real data, you're just as well off saying, "How many significant figures?" and the rolling percentile dice and taking that as your "guess".
You actually have a much better chance of picking the correct answer by complete random chance (again, assuming no data), because humans are wired to pick non-random numbers. I would guess that a huge number of people would guess 50-50, then 60-40, and then probably 7, 11, 17, 12, and other metaphysically laden numbers would be next most likely choices for people to select... but none of them is quantifiable as being "more likely" to be the correct answer than some random percentage.
The latest word from the OED, on its website and probably what you'll get in the next edition when it comes out -- FSM save us -- in non-paper, is that the 10^12 version "is now standard in the U.S. and is increasingly common in British usage."
Hey, we Amurricans at least know what we mean when we use the word. Well, anyway, the 20% or so who do know don't have to explain to each other which value they're using.
Incidentally, in 1969 I heard Tony Benn use billion in the American sense, 10^9, in a speech in Commons. I infer the sense from the fact that he was talking about North Sea oil reserves, and was not measuring in milli-barrels or anything. And it wasn't just leftie-usage back then: the OED finds a 10^12 trillion in the Telegraph from 1971.
All this long discussion, and I still don't know what the chances are of the LHC destroying the world...
At the moment? Zero.
It's been turned off.
Lack of data doesn't infer lack of foreknowledge. Personally, I'd tend toward a Gaussian distribution over other guesses on probability such as 50/50, seeing as how often that distribution appears in real life scenarios.
However, most people have at least knowledge of elementary arithmetic, in which there are a lot of basic assertions for a number of common events. Those of us who frequent Bruce's blog are probably a standard deviation (perhaps several) off of that norm. ;)
How many integers in the infinite series are even versus odd? 50/50. (Technically there is exactly one more even number than all of the odds, which is zero.) Negative vs. positive? If you were only asked to guess at whether the number I was thinking of in terms of even or odd, you'd have an equal chance. It's generally safe to assume that, should anyone ask you to pick a number between two arbitrary natural numbers, they're not going to expect you to be a shit and throw π, or e out as your guess, after all.
How about a child being born male or female? 50% How about the probability that it's night or day out at any given time?
Coin flips are known to everyone as are games of rock-paper-scissors to decide who begins a match (the latter being somewhat deterministic, as individuals tend to have biases).
People often think in terms of 'right' and 'wrong' or 'good' and 'evil', etc. These characterizations lead us to make a lot of assumptions, like that of a 50/50 split.
I think those ideas are pretty well ingrained in many cultures, such as the yin and yang of Chinese culture, for example.
In summary, I think it's perfectly reasonable to expect that the majority of people who one may encounter are relatively untrained in mathematics (and have forgotten much of what they might have once understood). It's also probably reasonable to expect that they'll have numerous cultural biases which shape the way they think.
Sadly, the upshot of the million/billion/trillion mess is that the terms are basically useless because they need to be qualified to something like "American trillion" or "UK billion" and then you have to know what *they* refer to.
I never use the word "billion" myself because of that niggling feeling of impreciseness as soon as you utter it.
You might as well just revert to an exponential notation and be done with it - at least it's unambiguous. The SI approach has some appeal too, but I'm in a country (Australia) which was sensible enough to go metric a long time ago so it seems pretty natural.
The same problem occurs with dates of course. There's no way of telling whether 1/2/09 refers to Jan 2 or Feb 1 because of the same transatlantic confusion so that notation might as well be abandoned as well.
Which is why people who want to be precise don't use dd/mm/yy or mm/dd/yy... they use yyyy/mm/dd. There's no month/day swap possibility in that order, and it has two other advantages: it makes yyyy/mm/dd hh:mm:ss a monotonic progression from largest to smallest units, and a trivial ASCII sort implementation will sort the dates in chronological order.
> In summary, I think it's perfectly reasonable to expect that
> the majority of people who one may encounter are relatively
> untrained in mathematics (and have forgotten much of what
> they might have once understood). It's also probably
> reasonable to expect that they'll have numerous cultural
> biases which shape the way they think.
I'm not sure if I'm misreading you, or you misread me, or we're both agreeing with each other, because that was sort of my point :)
Ah, I see... I mis-read the scenario that Nannite gave, my bad. Her example was:
"You and a friend are taken into custody and put into two separate jail cells where you cannot communicate with each other. You are told some unknown experiment is taking place and that there are two possible outcomes, A and B.
You are then forced to guess the probability of A occurring. Your friend is in the same situation and you must both guess the same value otherwise you will be killed."
In this case, you're right, what you are guessing is what your friend is guessing, not what the actual probability of A occurring *is*.
Well, in this case, what you're doing has nothing to do with statistics and probability distributions and everything to do with knowing what the other person would guess. This doesn't tell us anything interesting about probability or the "reasonableness" of picking "50-50" as a default guess for an actual probability distribution, though. Maybe that's why I mis-read the example :)
I think the weird prison cell thing would be better if the guy in the other cell was an incomprehensible alien. You are both trying to guess the probability of A, but the other fellow is a being of pure energy who experiences time in three dimensions simultaneously. What do you guess? The answer is that you have no idea. And this shows what the proper answer is if you remove the element of human psychology from the mix.
The "weird prison cell thing" is a classic game theory situation, the point of the lesson being that humans [whose backgrounds are similar] may be able to guess what each other would answer to certain kinds of problems.
Like Pat says, the game is not so much to match reality as to match the other person's guess.
@Nannite, Tangerine Blue: First of all, from a game theory perspective, that game is sort of vacuous. By definition there is nothing to base your decision on. Certainly 0.5 is no better than any other number between 0 and 1 based solely on game-theoretical considerations. It's only better because the conventional wisdom has it that two options go 50/50 by default. So if your partner is a probability ignoramus, perhaps you'd better go with 0.5. But not because of game theory.
Secondly, anybody putting me and a friend in that situation is an ass, and I'd have no trouble telling them so, too, because anyone who'd do that, I wouldn't trust them if they said they'd let me go if we did guess the same number.
Lastly, I don't think estimation by extortion is a good motivational example. People don't tend to think rationally when their life is in danger (so maybe I'm kidding myself when I say I'd tell this guy he's an ass). So no, I don't think this is "classical game theory," or at least I don't think that classical game theory has much to say about this game. Psychology, though, maybe.
@Ward Denker: You're quite right that we don't generally have a complete lack of data. Such scenarios are usually artificial.
The problem in the original story is that there is some data (or foreknowledge, or whatever you may call it), but the science teacher steadfastly refuses to use it. Or is not properly equipped to use it. Or both. At any rate, he lazily ignores whatever other information is available, and assumes that two options divide 50/50 by default. There is no default, except in people's imagination.
Gaussian distributions typically come about because of something in nature approaching the central limit theorem. So again, if I *know* that there's a sum of lots of iid variables, then I guess Gaussian, sure. But absent that, I have no reason at all to expect Gaussian. I have no reason to expect anything at all. It's not nihilist not to expect anything, if you really don't know anything. (But since you will practically never run into a situation of which you know literally nothing, this means it's important to be aware of exactly what you do know.)
By the way, there are exactly as many evens as odds. Arithmetic with infinite cardinalities don't work the same way as with integers. There's a one-to-one mapping between evens and odds whether you include zero or not. (If you include zero, use f(x) = x+1. If you don't, use f(x) = x+1 if x > 0, and x-1 if x
>Better still, yyyy-mm-dd, as per ISO 8601
Yes, this looks like the way to resolve the problem once and for all. I should just get into the habit of entering dates into applications as yyyy-mm-dd. I haven't encountered an application that gets that wrong.
>Which is why people who want to be precise don't use dd/mm/yy or mm/dd/yy
The point I was trying to make was that this is often the default display format for an application and it's not always possible to override it.
For example I entered the date 2008-12-1 into a Google Docs spreadsheet and it helpfully translated it to 12/1/2008 because I had defaulted to a US locale. I switched to a UK locale and it did the right thing and displayed it as 1/12/2008 but you can't tell at a glance what the date is without knowing that.
You pay teachers peanuts, you hire a bunch of monkeys. They're public employees so it's impossible to fire them for incompetence, so even the retarded monkeys continue to teach. Mr Walter L. Wagner's job is secure until he decides to quit.
Of course, there are some competent, dedicated high-school math teachers. But the probability of finding one in your local high school is ... well, let's say it's less than 50%.
Is there any way to view this video from Europe? The US site just gives a blank video player, and the Canadian link in the comments says I'm in the wrong region. I also can't see it on youtube.
@Petey: Good points. We would like to assume that a coin flip is not only independent of previous coin flips, but also unbiased (i.e. that on any particular flip, it will produce heads with 50% probability).
Now a couple of flips that all come up heads does not (statistically) say much about the behaviour of this coin. But a trillion flips in a row that all came up heads (or even a few hundred!) would seem to be STRONG evidence that the coin is NOT unbiased.
"By the way, there are exactly as many evens as odds. Arithmetic with infinite cardinalities don't work the same way as with integers. There's a one-to-one mapping between evens and odds whether you include zero or not. (If you include zero, use f(x) = x+1. If you don't, use f(x) = x+1 if x > 0, and x-1 if x
This made me smile. This is akin to saying "people don't fall down manholes if you put covers on them." It's certainly true, but it's not exactly the point. ;)
Who is Bruce Schneier to talk about mathematical illiteracy?
Mr. Schneier, let me remind you that you do not have a Ph.D. degree.
Silly Germans. A Billiarden is a pool hall.
@Ward: Of course it's not the point. That's why I was careful to preface that paragraph with "By the way." :)
You say "I'd tend toward a Gaussian distribution over other guesses on probability such as 50/50, seeing as how often that distribution appears in real life scenarios."
But I would bet a small sum that power laws are much more common. Which is why this conversation is happening on Bruce's blog not your or mine.
As for male/female gender split, at conception the ration is more like male 55% (we are less stable genetically than females apparently) and at birth around male 51% (we are also more likely to kill ourselves doing stupid shit than females.)
There is actually psychological reasoning why we can't tell the difference between very large numbers. It's related to the fact that we learn with a number line, and we tend to "see" numbers closer to zero easier.
This podcast explains more (I can't find a better link which is more.. readable.
I have a Ph.D. degree and I agree with Bruce! (Fortunately it is very hard to take a PhD away here;-)
Seriously, if people estimate things they do not understand at 50:50, that would explain why most have trouble with understanding the risks of rare events. Useful information and no little amusing.
Side note: This guy being a science teacher sounds fitting for the backwards, 3rd world country he is from...oh, wait.
"As for male/female gender split, at conception the ration is more like male 55% (we are less stable genetically than females apparently) and at birth around male 51% (we are also more likely to kill ourselves doing stupid shit than females.)"
The 'less stable genetically' assertion seems intuitively correct, given the speculation that the Y chromosome is naught but a mangled chromosome. There are certainly a number of sex-linked traits, and a lot of them are unfavorable for males.
The second (that we're more likely to kill ourselves doing stupid shit) seems less so. I admit a certain bias toward more concrete fields of science (biology) than the weaker social sciences (psychology) which, perhaps, taints my expectations on this matter. It just doesn't seem terribly plausible that the margin of "doing stupid shit" would be so statistically high, given evolutionary theory. Clearly some females are interested in that trait, generally at ages prior to childbearing (the flocks of girls who surround street racers come to mind), but it obviously wanes as responsibility and that intrinsic desire for security asserts itself. There's probably some bias in that data along those lines, but I'm not comfortable in an assertion that as high as 1/20 males die doing stupid things....
As for the first bit, I confess an attraction to that assumption. I'm currently reading "Prime Obsession" and see analytical concepts swimming about in my head. My proclivity is to take extra care to not see the entire world (though I certainly see the world a different way now, perhaps irrevocably) through whichever lens I've just picked up, so I'm perhaps over-leary of looking for relevance of power functions.
The elegance of notions such as π(x) ~ li(x) strikes me more, I think, because I've only recently taught myself calculus. Aside from calculating areas, I'm fascinated to see the integral pop up elsewhere (I expected it to, and am impatient to discover its application elsewhere). Its my current distraction from the nuisance that the practice of taking integrals themselves have proven to be. ;)
Bruce, actually this link is much more interesting than that, although it is funny.
This is fromt he Wall Street Journal:
"Today's simple truth: Half of all children are below average in intelligence. "
The interesting thing is that that is how averages ACTUALLY work. I think we know in which half Charles Murray belongs.
People ask me how I choose my lotto numbers. I tell them that I just pick last week's winners, and they look at me sideways. I reply that last weeks winning numbers have the same chance of winning as any other set of numbers. Then they shake their head about how silly I am and how I will never win, and tell me all about their strategy and birthdates.
@steve: On the few occasions I play (maybe a few times a year), I always have the machine pick my numbers. If I really was interested in maximizing my potential return, I'd probably pick high numbers, but still random in some way. The reason is wholly psychological: Although it makes no real difference, the emotional blow of seeing my numbers win on a week I *didn't* play would be substantial. In the case of your heuristic, if a repeat happened on a week I didn't buy a ticket, that would be a major bummer.
@Charles: I think it's clear that Charles Murray does understand that about averages. But I also think that IQ is a pretty crude way to assess people's intelligence. In my opinion, it measures the dot product between the test taker's mind and the test maker's mind, and little else. (I understand it's somewhat well correlated with future salary, though.)
You have failed to prove, of course, that the chances that the LHC can destroy the world are 0 when it is turned off. It may be that an unpowered LHC contains more dangers than a powered LHC.
In general, of course, people have forgotten to consider the post-black-hole-creation scenario: How do you comprehend "better", "worse" or even "destroy" in an environment where you cross a black hole boundary?
If a black hole event does occur, maybe heaven and earth as we know it will be destroyed and be instantly replaced with a better one.
At another level, what if we're crossing black hole boundaries all the time, but just don't know it?
The comments on how the number of even numbers equals the number of odd numbers, no matter how you treat zero, seems a little retrograde. After all, a rather more surprising result has been known for nearly 400 years: the number of perfect squares equals the number of whole numbers.
Again, think: one to one correspondence. The proof is not difficult.
This has been called Galileo's Paradox, a pretty fair eponym, since it appears in Two New Sciences (1638) and does not seem to have been stated with the same clarity by any earlier author.
I'm not sure why it is that the statement is surprising. Zero and one both behave the same way - being squares of themselves. Two and all numbers beyond have corresponding squares.
Is it because of the behavior of zero and one that gives people trouble imagining this? It's also not difficult to imagine if one is geometrically-minded: an actual square of any arbitrary unit in whole numbers can be constructed.
Zero and one are certainly naughty numbers, though. Paired up (for instance, in identity matrices in linear algebra) they wreak havoc. The proof of the PNT is concerned with the area between both numbers (the "critical strip") in which all of the nontrivial zeroes of the Riemann zeta function are proven to reside. They're also the only numbers one needs in order to represent any other number (the binary numbering system being the simplest).
As the "dynamic duo" of mathematics, I guess one should rather expect them to possess super powers. ;)
@Ward: I think if you had not learned to think of cardinality in terms of one-to-one correspondences, you'd find it mighty surprising that a set and one of its proper subsets had the same "size." (In fact, it was your comment about the "extra" even--zero--that led me to think you didn't entirely get one-to-one correspondences, and to write the aside.) I don't think it has anything to do with zero and one, specifically.
@Porlock: Evens and odds were on the table because besides having one-to-one correspondence, they also are equiprobable. That is, in the limit, as an interval of integers gets larger without bound, the fraction of evens and the fraction of odds both equal 1/2. That is, of course, not true of squares and non-squares.
The simplest disproof of this belief is one akin to Galileo's proof that objects cannot fall at rates that depend on their speed. Assume it's true, and show that it leads to two different answers to the same problem.
What's the probability that in the next ten years, a nuclear bomb will be detonated on only the western half of the United States? We don't know, so 50-50 is our best guess.
What's the probability that in the next ten years, a nuclear bomb will be detonated on only the eastern half of the United States? We don't know, so 50-50 is our best guess.
What's the probability that no nuclear bomb will be detonated in the United States? We don't know, so 50-50 is our best guess. And, of course, the probability of two bombs, one in the eastern half and one in the western is also 50-50.
So now we have four possible cases, all mutually exclusive, one of which must occur. And our best guess for the probability of each one is 50-50.
Similarly, our guess that at least one bomb will be detonated is also 50-50. So we have two events, each with a 50-50 chance. And the probability that both will occur is 50-50, the probability that neither will occur is 50-50, and the probability that one but not the other will occur is also 50-50.
Surely we can do better than these obviously nonsensical alleged "best guess"es.
That's why ancient greeks named 10.000 as "myriad" (numberless, countless, infinite): they knew commoners wouldn't understand bigger numbers. True than, still true now.
correction: True THEN, still true now.
@George: The Daily Show is a comedy show, not a cable news show.
Schneier.com is a personal website. Opinions expressed are not necessarily those of BT.