Taleb on the Limitations of Risk Management

Nice paragraph on the limitations of risk management in this occasionally interesting interview with Nicholas Taleb:

Because then you get a Maginot Line problem. [After World War I, the French erected concrete fortifications to prevent Germany from invading again—a response to the previous war, which proved ineffective for the next one.] You know, they make sure they solve that particular problem, the Germans will not invade from here. The thing you have to be aware of most obviously is scenario planning, because typically if you talk about scenarios, you’ll overestimate the probability of these scenarios. If you examine them at the expense of those you don’t examine, sometimes it has left a lot of people worse off, so scenario planning can be bad. I’ll just take my track record. Those who did scenario planning have not fared better than those who did not do scenario planning. A lot of people have done some kind of “make-sense” type measures, and that has made them more vulnerable because they give the illusion of having done your job. This is the problem with risk management. I always come back to a classical question. Don’t give a fool the illusion of risk management. Don’t ask someone to guess the number of dentists in Manhattan after asking him the last four digits of his Social Security number. The numbers will always be correlated. I actually did some work on risk management, to show how stupid we are when it comes to risk.

Tags: , , , ,

Posted on October 3, 2008 at 7:48 AM • 33 Comments

Comments

David • October 3, 2008 9:27 AM

He’s completely wrong about the Maginot Line. Divide the French front into the right (facing Germany), the middle (southern and eastern Belgium), and the left (central and western Belgium). In World War I, the Germans attacked along the front, most intensely and successfully on the French left.

Between the wars, the French built the Maginot Line on the right, to conserve on defenders. They put the best part of their army on the left, where the Germans had attacked in WWI and where early German attack plans in WWII were directed. They left weak forces in the center, where they considered the terrain very bad for attacking through (as shown in the Battle of the Bulge in 1944).

In 1940, the Maginot Line did its job very well, and the strong French and British left wing advanced according to plan. The main German attack was in the center; while the terrain was unfavorable for attacking, it was possible to move troops through it to attack at the French center. This led to the cutting off and the defeat of the strongest Allied forces.

The lesson is not that the French were overall stupid and unimaginative. The lesson is that they were defeated because the Germans were able to do what the French had considered impossible: launch an attack through the Ardennes Forest.

The worst security problems are going to come through completely unexpected attacks, similar to what the US military appears to be calling the “unknown unknowns”. The French guarded against any attack they considered possible, and were swiftly and decisively defeated.

John Scholes • October 3, 2008 9:35 AM

Hmmm. I am afraid I am one of his many detractors. He tends to say the obvious at inordinate length.

But then I was trained as a mathematician and tend to be fairly sophisticated about risk. It is certainly correct that the public is extremely unsophisticated about risk. Most people’s idea of how risky something is seems to depend almost entirely on recent media coverage. A distressingly large proportion of people also seem to be incapable of the simplest mental arithmetic.

So I suppose I should applaud his popularizing books … I guess the reason I don’t is that he rambles too much.

Kachda • October 3, 2008 9:44 AM

Quoting David:
The lesson is not that the French were overall stupid and unimaginative. The lesson is that they were defeated because the Germans were able to do what the French had considered impossible: launch an attack through the Ardennes Forest.

And thus you prove his point. Taleb’s contention is that you always plan for the expected, and then the unexpected thing which you never imagined/planned for, happens and wipes out your best laid plans.

Dave X • October 3, 2008 10:00 AM

Sounds like a fun phrase to apply to the TSA: “The TSA is a Maginot Line drawn in the sky.”

Brandioch Conner • October 3, 2008 10:09 AM

@Kachda
“And thus you prove his point. Taleb’s contention is that you always plan for the expected, and then the unexpected thing which you never imagined/planned for, happens and wipes out your best laid plans.”

And until that unexpected event happens, your “best laid plans” work.

Which is the part of the equation that he misses.

Along with the near infinite number of extremely rare and unlikely incidents that did NOT happen.

You’d be a fool to “invest” your retirement savings in the Lottery. The odds of you winning are almost non-existent.

Yet the odds of SOMEONE winning are fairly reasonable.

Josh Prismon • October 3, 2008 10:24 AM

“The lesson is not that the French were overall stupid and unimaginative. The lesson is that they were defeated because the Germans were able to do what the French had considered impossible: launch an attack through the Ardennes Forest.”

Just to follow up, that’s not completely true. The French had plans to extend the maginot line across the entire border from the coast all the way down. The Maginot line was fairly expensive, and plans for this were shelved with the Great Depression.

Thats why this concept doesn’t necessarily work for risk management. Warfare always involved a opponent trying to destroy you, versus risk management which needs to take into account more random events.

The french left the germans a opening, which the germans exploited. It was not a easy opening, as much as we love to mock the french, and the maginot line was just as effective as the Siegfriend Line and the atlantic wall.

T • October 3, 2008 11:02 AM

The greater issue and more important point is that if you cannot plan for everything, you handle it with:

  • Redundancies
  • Reserves
  • Deliberately under stressed / lightly loaded systems

Modern management thinking tend to place very little value on these concepts as it conflicts with “optimization” and “efficiency” and “maxi / minimization”.

Davi Ottenheimer • October 3, 2008 11:27 AM

@ Kachda

“Taleb’s contention is that you always plan for the expected”

That sounds like the philosophy that David Hume probably examined most famously in the 1700s.

In brief, you can never know how the next door knob will turn and yet you are able to open doors by applying experience from prior door knobs.

The Maginot Line gets a bad rap and is quoted ad nauseam, but David is essentially right. The line forced the Germans through a different front. In other words, the French put up a firewall and the Germans came through port 80. After a SQL injection attack are you going to say everyone’s plans for security have been wiped out? I don’t think so.

The bottom line is those who listen to all feedback, including naysayers and negativity, are the ones most prepared for the unexpected. Those who demand positive spin and a “can do” attitude at all times might as well march under a white flag.

Nick Lancaster • October 3, 2008 11:29 AM

In a meeting with company executives, one department head started talking about ‘patterned absences’ – i.e., employees who habitually call in sick on Fridays or Mondays or the day after a holiday.

He was so convinced that a) such patterns exist and b) they exist in our workplace that he couldn’t even process the idea that he was finding evidence to support a pre-conceived notion, rather than viewing absenteeism through an objective lens.

Randall • October 3, 2008 11:40 AM

“The lesson is not that the French were overall stupid and unimaginative. The lesson is that they were defeated because the Germans were able to do what the French had considered impossible: launch an attack through the Ardennes Forest.”

This is mostly a matter of semantics, but it seems there is a lack of imagination in assuming the Ardennes Forest would be enough of a natural fortification. Taleb’s larger point sort of applies, in that the French had a particular attack scenario in mind and weren’t prepared for others.

(Unless, of course, the French simply lacked the manpower, equipment, etc. to win, in which case we’d be second-guessing whatever strategy they came up with, even a brilliant one.)

But it is interesting that the Maginot Line had a more subtle purpose than most of us realize today, and your point that “impossible” attacks are a major threat is dead on.

Eadwacer • October 3, 2008 11:58 AM

Another reason the French lost is that they learned another, wrong, lesson. In WWI it turned out that, while no attacks worked very well, the ones that worked best were the ones that were carefully scripted and centrally controlled. When things came apart in WWII, the French command system couldn’t respond in time. Lesson for today: the ability to respond to the unexpected requires flexibility of response, not scripted reactions.

Matthew Carrick • October 3, 2008 12:03 PM

Let me remind folks that some 10 days(?) into the German invasion of France & the Low Countries a small outpost on the Maginot Line was taken by assault. This prompted the French to divert sorely needed mechanized forces from the center (where they were reasonably well placed) to defend the Maginot Line! So, a successful diversionary attack can allow an otherwise limited attack to succeed out of proportion to its initial aims by pinning enemy units.

David • October 3, 2008 12:52 PM

@T: Excellent point; another reason the French lost was the lack of adequate theater reserves. This ties in nicely with Bruce’s continued insistence that we need to be able to react adequately to terrorist attacks and disasters, since we can’t stop them. Eadwater’s comments are also relevant here.

What I most dislike about the misuse of the Maginot Line argument is that it implies that the French were stupid, that any of us could easily have done better. The mistakes the French made were more reasonable. They didn’t extend the Maginot line for economic and diplomatic reasons. They didn’t have a more modern command and control system because they didn’t consider it as high a priority as other things. They had an inadequate reserve because they had too many things they wanted to accomplish with too little force. They erred in their terrain analysis.

The takeaway from the 1940 campaign should be that you can do everything reasonably and still get blindsided, not that you can get blindsided because of your own stupidity. Just because we’re smarter than the French command stereotype doesn’t mean we’re immune to exceedingly unpleasant surprises.

Steven Hoober • October 3, 2008 1:21 PM

Everyone’s said it in bits and pieces, but I wanted to wrap the Maginot line issue directly to security. Cause, you know, border security in a war is right up there.

Yes, the line was supposed to be all the way across. And was killed due to money. But command, control and communications (including transport) across the frontier was also dropped, and more importantly, /reaction/ forces. Not just “reserves,” but full-strength, full-modern, mechanized front line troops in place to do something about a breach of the fixed security.

So, for money, or customer experience, or executive whim, you have to leave a hole in the fence/firewall/bunker-line. And it’s a pretty obscure, difficult-to-traverse hole that historically no one has used before. When the decision is made to drop it, everyone is sure it’s a problem, so they make sure to leave a camera up there and have the number for the police close at hand. Or have scouts, and reaction forces.

But by the time the germans cross it, budgets keep being cut, and there’s no one looking that closely, few forces to respond to it and none that are up to the task.

Each decision to drop a piece of the force structure was made independently, and probably seemed reasonable at the time. Mostly, there were indeed contingency plans and stopgap measures, and promises of fixing it in the next release, I mean fiscal year. But the net effect was exceedingly non-trivial.

Sounds pretty familiar, and therefore a pretty apt comparison, to me.

Chris S • October 3, 2008 2:49 PM

“The numbers will always be correlated.”

Always???

Let’s see….
8000000 people in Manhattan
0.5 go to dentists = 4000000 people
2 times a year = 8000000 visits
for 50 weeks = 160000 visits/week
for 5 days = 32000 visits/day
for 1/2 hour = 16000 hrs/day
from a 5 hour working day = …
3200 dentist-days/day

…implying 3200 dentists in Manhattan.

Disagree with my assumptions if you have better information, but I wouldn’t change this number just because I had previously thought of some other number.

Jess • October 3, 2008 3:21 PM

Chris,

If you really needed to know that number, why wouldn’t you just look it up? It took me 20 seconds to google to http://www.op.nysed.gov/dentcounts.htm and discover that the answer to your question is 2,506. Which isn’t far off from your estimate, but if I were spending money on something I’d like to use the real number. Also, it’s clear that this method was faster than yours.

Taleb’s larger point is to keep your eyes open, and never let your model blind you to the real world.

Jess • October 3, 2008 3:55 PM

The interview misrepresented the Mediocristan/Extremistan concept somewhat. Mediocristan is not “a bogus model of reality”. It is one of the two modes of dynamism that Taleb identifies in real-world situations. The error that many people make is to perform Mediocristan-appropriate analysis when they’re really in Extremistan.

Like these idiot bankers.

Jess • October 3, 2008 3:58 PM

One more point and I’ll shut up: I know a bunch of dentists, and I can’t think of one of them who works 5 days a week.

This morning my father was complaining about 4 days a week.

Yeah, those dentists…

Chris S • October 3, 2008 4:01 PM

Jess;

Normally I would look it up. But the point in the interview was in regard to an answer where I guess, not where I actually research it.

So, I guessed. Granted, I guessed the individual assumptions, and then did a calculation.

But his examples seem to at times detract from his ideas, rather than help clarify them.

Two things I noted about the interview – he comes across in this article as opininated. I get the impression that he might be prickly to get along with just because of the way be behaves. He comes quite close to expressing this — “I’m too messianic in my views.”

The other, and much more important, item, is that the core of what he’s discussing is neither new nor undiscussed if you’ve been looking. Think of Knuth’s comment on premature optimization. Or go search out Bart Kosko’s views in The World Question Center in 2006, 2007, and 2008. Or go research the term ‘simplexity’.

These are valuable ideas, and I’m glad they are getting a greater airing. I would be little happier if the people presenting them didn’t come close to ranting at times. I would be a LOT happier if it wasn’t taking a global financial crisis to bring these views out more.

Shachar Shemesh • October 3, 2008 4:23 PM

If you want to talk about risk management and being blinded by preconceived scenarios, and with Yom Kipur next week, I’d go for the Bar-Lev line over the Maginot line.

In 1967 Israel fought (among others) Egypt, and conquered the Sinai desert. The entire war was won in 6 days, with the entire Egyptian air force destroyed during the first few hours of the war. Between 1967 and 1973, the Egyptian prime minister has changed (from Nasser to Anuar Sa’adat). The Israeli intelligence claimed that the show of force that the six days war was is enough of a deterring force to keep the Arab countries from attacking. The intel also referred to Sa’adat as a peasant who values the land, and who will, therefor, not give up the idea of capturing the Sinai desert back.

In order to keep the line clean, a line of holds was erected along the Suez channel, called the “Bar-Lev line”. The idea was that, in case of an attack, this line will be able to fend the Egyptian forces back for long enough for the reserve forces to assemble and get into battle.

In October of 1973, intel detected increased military activity along the Egyptian border, but discarded it as an exercise because “The Egyptian will not attack when they know they cannot get the Sinai dessert back”. This despite an Israeli spy who was a high ranking officer in Egypt warning of an attack, and despite having Hussein, king of Jordan, fly to the border and personally meet Golda Meir, prime minister of Israel, to tell her that Egypt and Syria are about to attack!

During Yom Kipur of 1973 the Egyptians did, in fact, cross the Suez channel. The Bar Lev line was unmaintained, due to Israel certainty that the Egyptians won’t attack, and did not hold the Egyptian forces back.

Attacking on Yom Kipur was, in fact, a poor choice on the Egyptian/Syrian side – attacking on a holiday where the entire country is at home and the roads are literally deserted for 24 hours meant that reserve forces deployment was much swifter than it would on a normal day.

With history’s hind sight, we can tell that Sa’adat approached Israel (through Henry Kissinger) some time before, and offered piece. His offer did have a “take it or leave it” side to it, requesting the whole of the Sinai dessert back. Even Kissinger told him that as the losing party to the war, Egypt was in no position to unilaterally set terms. Sa’adat did not start the war to win Sinai back. He started it to get the Israeli’s attention. As history can tell, only 6 years later, a piece agreement was reached which was, more or less, precisely what was originally offered by Sa’adat and rejected by both Israel and the USA.

Shachar

Clive Robinson • October 3, 2008 4:25 PM

@ Eadwacer

“Lesson for today: the ability to respond to the unexpected requires flexibility of response, not scripted reactions.”

Actually not quite. It requires predictable results from subordinates on commands. That is “drill”, you train people to do certain small tasks in predictable ways. The key is ensuring the small tasks can be linked together to acomplish large tasks. You then train people to do the job above theirs, if they can do it you promote them. At a certain level you also train people to make informed decisions so that they can respond flexably.

Or in IT terms 😉 In essence the troops supply the “OS calls” middle managment provide the “program subroutiens” and senior managment the “top level and control loop”.

With regards to the Maginot Line, one of the main reasons that the Germans defeated the French was that they actually did three things,

1, They learnt the lessons of WW1.
2, They tested the knowledge with war games (Spain etc).
3, They took the time to apply those leasons to their troops.

The Germans realised that the secret to modern warfare is actually quite simple,

1, Fast mobile troops.
2, Fast Command and Control.
3, Realise that troops and armour/artilery need to work hand in hand not individualy.
4, develop and deploy new methods effectivly.

The French however did not prepare for war nor did the rest of Europe even though the warning signs where there.

The reason was simple Europe was disilusioned with politics by combat.

Due to certain decisions made during WW1 (Pals brigades or join together stay together recruitment in Britain) the effects of the fixed warfare took a very high toll on the civilian population. This resulted in a very embittered out look amongst the population and polititions knew (as Churchill found out) that the people could not stomach rearmament and where uterly opposed to the idea of war. Similar sentiments where to be found across all the WW1 “victor” countries. Similar sentiment was initialy felt in Germany but the effects of the reperations (compensation) Germans had to pay caused starvation, demoralisation and desperation.

One of the reasons Hittler was popular was he put hope in hearts, bread in stomachs, jobs to give self respect and rhetoric to rebuild dignity and easy escape goats (gypsies, jews, non-arians, deviants, the mental and physicaly weak etc, etc) to blaim for the comman Germans misfortune, and no surprise they lapped it up. The reason the rhetoric worked was the language used always sounded reasonable…

It was to prevent this series of events re-occuring that the Marshall Plan was put into action.

It is a lesson we should all learn from, as the current economic climate will give easy oportunities to the next set of naredowells with political agenders they wish to foist on the masses.

As once noted by the 3rd U.S President,

“The condition upon which God hath given liberty to man is eternal vigilance…”

Jess • October 3, 2008 4:27 PM

Knuth is great, but if you think he’s covered all of Taleb’s ideas (ANY of Taleb’s ideas? really?) you ought to read more Taleb. He would be the last person not to acknowledge a previous thinker’s influence. The running gag on his homepage recently has been the complaint that the problem of induction shouldn’t be named for Hume, but rather for Bishop Huet. Then more recently he has thrown over Huet for La Mothe Le Vayer. He regularly references Sextus Empiricus on this topic as well.

The problem is not that no one has ever heard of empiricism, but rather that no one alive today has ever practiced it.

John Campbell • October 3, 2008 9:54 PM

Offense vs Defense… who has the initiative?

When you surrender the initiative– and lose sight of a goal– you go on the defensive.

Leadership maximizes gains whilst management tries to minimize losses.

More • October 3, 2008 10:28 PM

@steve hoober

The process you are describing is happening to American institutions all over.

Resources are being diverted to “must do” things, like the Wall Street bail out, the ongoing war in Iraq and Afghanistan, etc.

Meanwhile, the basic tasks, like maintenance, repair, replacement of worn out items (and people) are pushed off to the side.

Incremental budget cuts further degrade the capabilities of the institutions even as vast sums are spent on contractors to do the work at a higher cost than in house staff.

As the economic pressures increase, the process accelerates….

Before you know it, the US ends up with their own Maginot lines.

The best example of this is airport security with federal employees.

They deployed a vast, expensive, and largely ineffective phalanx of screeners for a threat that may never occur again.

Frogman • October 5, 2008 10:18 AM

It may prove intellectually satisfying for some to pick at Nick Taleb’s examples and show that he’s often somewhat to nearly completely wrong in his assertions. But as so many above have asked, so what? So what does that have to do with the central and crucial argument of Taleb’s work? Which is, like it or not, Black Swans exist and there’s not a whole lot we can do about them.

Look, I get it that accepting the inevitability of the Black Swan–the inevitability of the “unknowable but inevitable”–must be disconcerting to security planners. But let not your hearts be glum.

Taleb’s not suggesting that you throw up your hands, hang up your hats, call it a day. He’s simply telling you that, in the long run, failure is not an option, it’s mandatory.

This is a notion those of us with a liberal/progressive bent find hard to accept. We believe in perfection. We think all is knowable. And if things aren’t perfect and if there is anything out there we don’t know, it’s only because we haven’t tried hard enough or–and this is the default mode these days– George W’s to blame.

Taleb’s suggestion for dealing with this? Live your life. Do the best you can to hedge against bad outcomes. Learn to distinguish between real experts and charlatans (that is to say, a good plumber versus a Nobel-prize winning economist). Take off the tie. Enjoy a good glass of wine. But don’t be surprised if a Black Swan shows up and bites you on the butt. (Isn’t that right, Roy? see pg. 130, The Black Swan.)

Clive Robinson • October 5, 2008 5:54 PM

@ Frogman,

“I get it that accepting the inevitability of the Black Swan–the inevitability of the “unknowable but inevitable”–must be disconcerting to security planners.”

Only to some.

I think you are indirectly refering to the,

There are known knowns,
Known unknowns,
And unknown unknows.

That one of George W Bushes side kicks put in a speech.

Well this is not new and there is a sensible security response to it that has been known since before the computer. Which is the,

Detere, Detect, Delay, Respond.

Mantra of physical security.

That is you design a system to discorage the average attacker from trying. You use reasonable methods of detecting attacks. Your system is designed to make the attackers progress slow, and you ensure that you respond to attacks in a very short time frame.

This works because most of the time the attackers goals can be identified before the system is designed.

Occasionaly an attacker will have a goal that is either unknown or considered to “off the wall” to have been designed against, but even then there is a mitigating stratagy (ie insurance) in most cases.

The hard parts are designing enough flexability into your system so that it has a degree of future proofing and designing it in such a way that it does not get in the legitimate users way whilst doing it at a reasonable cost.

“Black swans” like meteorites do come along but it’s only a truly exceptional one that does not either get burned or else lands harmlessly.

Lawrence D'Oliveiro • October 5, 2008 6:29 PM

Jess said: “The problem is not that no one has ever heard of empiricism, but rather that no one alive today has ever practiced it.”

You mean, you don’t know of anyone who has practised it.

bob • October 6, 2008 7:28 AM

The Maginot Line was a tactical success but a strategic failure.

This guy is right by the only scorekeeping system we currently have: he has made a lot of money.

windscar • October 8, 2008 5:19 AM

If we don’t listen to Taleb, we’ll blame each other every time a terrorist attack succeeds. That gives every admisistration incentive to do anything they can to prevent an attack, even if it means taking away our rights and spending all our money.

We’ll be as safe as possible for the few years until we run out of money. Then we’ll lose, horribly.

thomas • November 15, 2010 9:17 AM

Would anyone ghave the original reference for the “dentists in Manhattan” priming study? many thanks!

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.