Threat Modeling at Microsoft

Interesting paper by Adam Shostack:

Abstract. Describes a decade of experience threat modeling products and services at Microsoft. Describes the current threat modeling methodology used in the Security Development Lifecycle. The methodology is a practical approach, usable by non-experts, centered on data ow diagrams and a threat enumeration technique of ‘STRIDE per element.’ The paper covers some lessons learned which are likely applicable to other security analysis techniques. The paper closes with some possible questions for academic research.

Tags: academic papers, Microsoft, risk assessment, threat models

Posted on October 13, 2008 at 6:21 AM • 17 Comments

Comments

Jeroen • October 13, 2008 7:02 AM

The first paragraph raises a red flag with me:

“a set of processes applied to all Microsoft products with signiﬁcant security or privacy risks.”

If anything, the past few years should have taught us that in software, security holes can occur in the most innocuous places, and not just in places that we may think of as “high-risk”.

A nonny mouse • October 13, 2008 7:28 AM

@Jeroen

While security holes can occur in the most innocuous of places, they (assumedly) most likely occur in high-risk places. It makes sense to me that you’d examine them in order of risk. The time and effort that can be invested in security is limited, so you have to make the most of it. You can’t do that if you start looking for security holes at the most unlikely places first.

Dom De Vitto • October 13, 2008 7:52 AM

Sounds like the paper is a cut from from the MS THreat Modelling book, which is (surprisingly) good….

Dom

Erik W • October 13, 2008 8:18 AM

Finally, an answer from Microsoft to the popular question “What the heck were you thinking?!”

Joe • October 13, 2008 8:51 AM

“centered on data ow diagrams”… That’s a great typo. In the paper it’s flow diagrams, but I like that better:

“you just opened an attachment from someone you’d never met before because it claimed to be pictures of nekkid women. Ow! Ow! Ow!”

Pete Austin • October 13, 2008 11:16 AM

Re: “Advice we gave, such as ‘think like an attacker,’ was meaningless to many people 4. I find challenging security experts to think like a professional chef helps illustrate the feeling.”

Finally I understand all the squid blogging.

Ragib Hasan • October 13, 2008 12:40 PM

Agree with Dom … Microsoft’s threat modeling book was quite good.

But I’m not convinced that you get to find out all the threats a priori … sometimes attacks happen in the most unimaginable manner.

wiredog • October 13, 2008 12:49 PM

Speaking of Microsoft, from today’s Security Fix at the Post:
Microsoft Stock Price Routinely Dinged by Security Patches
http://voices.washingtonpost.com/securityfix/2008/10/microsoft_stock_price_routinel.html#comments

“the other predictable part of Patch Tuesday and Advance Notification Thursday is that Microsoft’s stock price almost always sinks on those days relative to other trading days.”

Which is odd, but does demonstrate a relationship of some sort between security and stock price.

Clive Robinson • October 13, 2008 1:37 PM

@ Ragib Hasan,

“… sometimes attacks happen in the most unimaginable manner.”

As far as I can tell most users who have had the “blue screen of death” have discovered a potential attack vector.

Thankfully most users lack the curiosity, time and dogged determination to move it on to a real live exploit 😉

Seriously most bugs are only found by logic reasoning post discovery.

Which is frightening when you consider just how many states even the most mundain bits of software can be in and just how few get excercised during testing…

interestingly even mathmaticaly you would expect to find more bugs with random input than by chosen input.

25 years ago I used to tell students 40% of your code should do what the spec calls for the other 60% should be for input errors and exceptions. I don’t think the lesson has yet been learnt 8(

Alex • October 13, 2008 7:02 PM

“I’m not convinced that you get to find out all the threats a priori … sometimes attacks happen in the most unimaginable manner.”

It gets worse. To understand what contributes to a loss event, not only do we have to understand the force applied by a Threat Community, but also our ability to resist it. So there are potentially “black swans” (unknown, unknowns) both in the Threat Capability, but also in the Resistance Strength.

Fortunately, there just aren’t a lot of Black Swans (in the traditional sense, not in Taleb’s re-definition).

Adam Shostack • October 13, 2008 10:05 PM

Jeroen,

Thanks for the comment. We take a fairly broad view of ‘significant risk.’ It includes anything internet-facing, possibly internet facing, or anything that is likely to store sensitive data.

And while holes can show up in unexpected places, as the nonny mouse suggests, it makes sense to start where you expect problems.

Adam Shostack • October 13, 2008 10:06 PM

Dom,

As the paper says, we learned a lot from all the experiences, including those that Window and Frank wrote about. Their book came out in 2002, we’ve learned a lot since then, and wanted to share.

Adam Shostack • October 13, 2008 10:37 PM

Alex,

I think that looking for Black Swans is all well and good, once you’ve found the white ones, and also taken stock of the chipmunks, skunks, and deer. A brief examination of just about any app will find a slew of ‘boring’ issues which need fixing, ranging from poor input validation to bad authentication to lack of integrity controls.

Jim T • October 14, 2008 12:27 AM

And the metadata tells us…”TeX”, not MS Word.
And Captain Obvious chuckled quietly in the background….

Winter • October 14, 2008 12:57 AM

“Threat Modeling” seems like winning the last war to me.

Threat modeling will tell you all the obvious attacks. It also tells you what tools and efforts are available.

I understood that you should use this knowledge to start mapping vulnerabilities.

The best lock in the world fixed into an armored door won’t help if the walls are made of wood.

And that seems to me the problem of Microsoft. As the foundations of Windows are build on, eg, ActiveX and running all services over sockets, what use is trying to secure the next little program.

Winter

Dave • October 14, 2008 8:20 PM

Having been through a threat model assessment or two at Microsoft, I think the best part was that you’d draw your design up on the whiteboard and separate safe areas (i.e. system data) from unsafe areas (i.e. user input). Very knowledgeable people would then spend an hour thinking of possible ways to get into the safe areas from the unsafe areas.

For every possibility they thought of, you as the designer had to show how the threat was mitigated (ie, we scrub this input for SQL, or they’d need to be running code on the server already in which case you’re already sunk).

You also fill out a lot of forms.

Brian • October 15, 2008 11:16 AM

“Given the very large number of security classes, tools and techniques available within Microsoft, it is reasonable to assume that most practitioners have at most, 2 hours of training in the last few years.”

Nice ‘n subtle. 😉

Threat Modeling at Microsoft

Comments

Leave a comment Cancel reply