Microsoft Windows Receives EAL 4+ Certification

Windows has a Common Criteria (CC) certification:

Microsoft announced that all the products earned the EAL 4 + (Evaluation Assurance Level), which is the highest level granted to a commercial product.

The products receiving CC certification include Windows XP Professional with Service Pack 2 and Windows XP Embedded with Service Pack 2. Four different versions of Windows Server 2003 also received certification.

Is this true?

…director of security engineering strategy at Microsoft Steve Lipner said the certifications are a significant proof point of Redmond’s commitment to creating secure software.

Or are the certifications proof that EAL 4+ isn’t worth much?

Tags: , , ,

Posted on December 20, 2005 at 7:21 AM • 48 Comments

Comments

mpd • December 20, 2005 8:00 AM

With Windows XP SP 2, you can create user accounts with Administrative rights but no password. You don’t even get a warning about it.

So I guess I don’t think much of EAL 4+.

Gustavo Bittencourt • December 20, 2005 8:10 AM

Are the Protection Profiles of theses certifications public?

There is much marketing in EAL level, but the Protection Profiles and Security Targets is what really cares.

Ah Clem • December 20, 2005 8:28 AM

I remember at the first CC conference where Microsoft talked about NT being certified. It was after they excluded the gui, notepad, IE (Note: Bill had just testified that IE was a part of the OS to Congress one month earlier), Outlook express, almost every application, etc, etc.

I am a big supporter of the CC, but you really have to take a good look at the protection profiles to see if they did a good job. Also, did they use the new version of CC?. The CC is like any standard, it can be an indication of security and it can be wimped out.

RSaunders • December 20, 2005 8:30 AM

The CAPP definition says it all:

The CAPP provides for a level of protection which is appropriate for an assumed non-hostile and well-managed user community requiring protection against threats of inadvertent or casual attempts to breach the system security. The profile is not intended to be applicable to circumstances in which protection is required against determined attempts by hostile and well funded attackers to breach system security. The CAPP does not fully address the threats posed by malicious system development or administrative personnel. CAPP-conformant products are suitable for use in both commercial and government environments.
(See http://niap.nist.gov/cc-scheme/pp/PP_CAPP_V1.d.html)

Sounds like they have solved a pretty low level problem, highest commercial claim notwithstanding.

Roy • December 20, 2005 8:37 AM

Remember the C2 compliance “helper” in NT 3.5? You only pass if NT is the only OS on the box (with 0 boot delay, even though that didn’t prevent getting the boot options menu), there’s no floppy drive and no network. (of course, back in those days, they didn’t even consider bootable CD-ROMs)

AG • December 20, 2005 9:14 AM

Exchange 2003 SP1 http://blogs.technet.com/exchange/archive/2005/12/12/415945.aspx
also received EAL 4

My understanding of the EAL Certs comes from the Ehlo Blog
Level 4 is “Methodically Designed, Tested, and Reviewed”

That does not mean it is secure or assured… just documented.

EAL level 5 is “Semi-formally Verified Design and Tested” which mean barely tested.

I personally have a Windows XP system SP1 that if I apply SP2 the machine blue screens and dies.
A service pack that makes your hardware out of date just plain sucks.

It is obvious TO ME that Microsoft engineered the test, demonstration, etc under perfect conditions to guarantee their results. The engineer test probably had a team who’s goal was;
“Be sure these products do not fail to meet EAL 4 Cert.”
when the true goal should be “Do these products meet EAL 4?”

Microsoft is VERY self centered they do not care about outside certs or standards. Also, security matters very little at Microsoft since their technology is always so ahead of the curve. Since they are so ahead of the curve they do not encounter the same issues we do day to day.

Ray Potter • December 20, 2005 9:24 AM

A Common Criteria evaluation (against CAPP or not) will not deliver a secure product. It will deliver a measure of confidence that the Target of Evaluation’s set of security functional requirements implemented counter a series of threats and complete certain objectives.

The Protection Profile is the embodiment of threats, assumptions, objectives, and requirements defined by an end user. And one thing people need to remember about the CAPP … Many government customers require compliance to that Protection Profile. Microsoft should not be blamed for pursuing a PP that may not be the end-all and be-all of operating system security. They are meeting (and exceeding, btw) the needs of their many end users who require a Common Criteria evaluated OS against the CAPP. The procurement policies stipulating adherence to the CAPP may not be the best solution, and one can make a safe bet that end users aren’t running in evaluated configuration anyway. But that’s an entirely different issue.

The CC can help a vendor create more secure software by defining a set of assurance measures (e.g., creating LLDs, detailed test plans, etc.) against security functions. Those who have been through CC @ EAL4+ know that it is not to be taken lightly. It is a very big deal, and MSFT should be proud to market their activity.

I wonder if similar dubious comments will be made when *nix platforms pursue this PP.

Faisal N. Jawdat • December 20, 2005 9:53 AM

On paper, NT should be much more secure than any of the *nix based consumer OS’s (ACL’s, roles, administrators who aren’t all-powerful, better logging, etc.). On paper.

From what I can tell, most government criteria with regard to software seem to focus on checklists that can be carried out by trained pet rocks, so it’s not surprising that they’d focus on documented capabilities over real world behavior.

Tombot • December 20, 2005 10:25 AM

Ray Potter OTM – a PP is written however the vendor wants it to be written. In a few cases industry groups have authored their own protection profiles for specific kinds of devices, but the US government doesn’t publish its standards in that format (to my knowledge). Vendors have to adapt NSA/DOD guidance to their own products, and of course the extra cost of achieving EAL4+ is just passed on to the customer (you, the taxpayer). CF $1200 PS/2 KVM switches and the like.

The CC is basically a failure. In trying to avoid creating a technology-dependent model that specifies safeguard/threat relationships, they wound up with something basically meaningless.

to Faisal: “documented capabilities” and “real-world behavior” are supposed to be the same thing – when they don’t match, we call it an Integrity issue. Which is exactly what the CC has given us.

Ray Potter • December 20, 2005 10:34 AM

Tombot- slight correction: A Security Target is written however the vendor wants it to be written, assuming it does not follow a Protection Profile. Even this isn’t true in all cases, as end users might provide input to vendors on items to include in their Security Target.

The US Government does publish Protection Profiles. You’re correct in saying that industry groups have authored PPs, but the most prolific writer of PPs is the US Federal Government. See http://niap.nist.gov/cc-scheme/pp/index.html and especially http://niap.nist.gov/pp/draft_pps/index.html. The latter lists all US Government Protection Profiles currently being developed.

Regarding the extra costs being passed on to the customer, I made this comment on the TaoSecurity post: “This is not the case. I ran the security assurance program for one of the most active product vendors in the FIPS 140/Common Criteria certifications space, and there is little to no room to add cost-recovery mechanisms to certified products. Why? Well, having certification is the ante to sell certain products to Government. Past that, the potential buyer will look at performance, interoperability, and cost. Product vendors can not and do not typically increase costs to products sold to government to recover certification costs because the market (e.g., a competitor) does not. The cost essentially materializes as the ‘cost of doing business’.”

Fred Page • December 20, 2005 10:36 AM

@AG:
“…security matters very little at Microsoft since their technology is always so ahead of the curve. Since they are so ahead of the curve they do not encounter the same issues we do day to day.”

I’m curious as to what you mean by “ahead of the curve”. Most Microsoft products I’ve used are behind at least 1, and sometimes several of their competitors technologically; this includes their office package, their OS, MFC, their compilers, their internet browser, etc.

Francois • December 20, 2005 10:44 AM

“On paper, NT should be much more secure than any of the *nix based consumer OS’s (ACL’s, roles, administrators who aren’t all-powerful, better logging, etc.).”

Agreed. My view is that NT has a good security model but poor implementation. On the other hand, Unix’s security model isn’t as good, but the system has the potential for much better implementation.

In order to reach a good level of security with NT, you end up removing a lot of functionality, possibly breaking it – i.e., a workstation with no floppy drive and no network connection.

However, any system’s security depends on the administrator’s capabilities, business needs, applications involved, defense-in-depth, end users, etc. It’s very possible to have an environment where a particular Windows workstation is, generally, a lot more secure than the Unix workstation next to it. It just all depends on what you’re doing.

Just my $.02.

AG • December 20, 2005 10:49 AM

@Fred Page
For Microsoft it is common practice to be involved in multiple pilots of new technology.
For instance your desktop will be running Office 12, your mailbox on Exchange 12, and your desktop Windows Vista.
With this type of setup Microsoft will not be affected by the viruses and security issues that we see in the real world everyday.

Tombot • December 20, 2005 10:56 AM

“Product vendors can not and do not typically increase costs to products sold to government to recover certification costs because the market (e.g., a competitor) does not”

Well, you have more detailed and probably braoder experience than I, but my “typically” (with regard to EAL4+ products) has been quite different. Where does the money to pay for certification come from? It’s not necessarily pertinent to all this but frankly CC Evaluation is a highly expensive process that keeps a lot of competitors OUT of the business of selling to the Federal government. I’ve talked to developers who’ve given up on having federal agencies as customers because they can’t put the time and cash up front to go through the hoops.

The NIST/NSA PPs only go up to EAL4 in a few cases, and only up to 3 for Operating Systems, written in 2001.

Ralph Broom • December 20, 2005 10:58 AM

The key to any CC certification is the Security Target (ST – where vendor claims are made), the Protection Profile (PP – optional, where additional requirements are specified) and the Validator’s Report (VR – where issues can be brought to the attention of potential users). All these are publicly available on the certification web site. The EAL level indicates how strenuously the ST claims have been evaluated, nothing more, nothing less. Commercial products have been evaluated up to EAL7.

Tombot • December 20, 2005 11:07 AM

Skim to the end of the ETR Part II Prop.doc downloadable from Chris Walsh’s link and you can find some sensitive-but-unclassified concerns on the part of the evaluation team regarding the writing of the target and the scope of the eval.

Additionally, the entire penetration testing portion seems to assume that eveyone will always follow the rules in Microsoft’s secure administration guidance – which goes right out the door once you need to run ActiveX for any purpose, among other things.

Ray Potter • December 20, 2005 11:13 AM

@ Tombot

“Where does the money to pay for certification come from?”

Most commonly it comes from the product vendor’s OPEX budget. If the business case is justified, the product vendor will spend the money and dedicate the resources because of the potential sales (usually > 3x the cost of the evaluation for it to be worthwhile).

“CC Evaluation is a highly expensive process that keeps a lot of competitors OUT of the business of selling to the Federal government.”

This is absolutely true and is an unfortunate byproduct of a complex and time-consuming evaluation methodology such as the CC. CC evaluations can cost hundreds of thousands of dollars, and it is possible for a vendor to be locked out of procurement because they cannot afford the cost. But I’ve also found that even small companies can come up with the dollars if they see justifiable business case.

Cerebus • December 20, 2005 11:33 AM

CAPP is worthless, IMHO, because it assumes that the environment is nearly totally benign and that there is no insider threat. Both of these assumptions have been shown, time and time again, to be false.

A more proper PP is “Single-level Operating Systems in Environments Requiring
Medium Robustness” (SLOSPP-MR).

Davi Ottenheimer • December 20, 2005 11:36 AM

“Or are the certifications proof that EAL 4+ isn’t worth much?”

Sure they’re hundreds of thousands of dollars, but the real cost was probably to train their engineers to be able to say “we keep security in mind at all times” and still keep a straight face.

Eric K. • December 20, 2005 12:09 PM

@AG:
“A service pack that makes your hardware out of date just plain sucks.”

Yeah, tell me about it. I can’t seem to get any flavor of Linux running on my abacus! Sucks, I tell you!

jammit • December 20, 2005 1:33 PM

< snarf>
Was MS running their OS in a Bochs shell under linux?
< /snarf>
I have a question. Were the operating systems in question a default setup (installed from CD on a new system) and configured for the network, or were they custom installed by MS engineers from source? As an example, on a normal install, IE is “part” of the OS and installed with it. On a custom install from source, the MS engineers could basically compile the OS without IE installed and remove a weakness of IE without actually fixing it. The XP embedded is an example of this. If you compile it so it’s locked out completely, and burn it to a read only ROM, you can’t change the ROM by infecting with software. It’d be like erasing an AOL cd for reuse.

Luís Bruno • December 20, 2005 4:49 PM

As a Mac user I might be biased against Microsoft. In case you haven’t had opportunity to read it, Shapiro’s assessment is available online at: http://eros.cs.jhu.edu/~shap/NT-EAL4.html

According to NIST[0], the current Windows certification is still a CAPP/EAL4 certification as Shapiro has analyzed, with the ALC_FLR.3 augmentation. The latter can be read at http://niap.nist.gov/cc-scheme/cc_docs/alc_flrv11.pdf, and deals with (security) flaw handling.

[0] http://niap.nist.gov/cc-scheme/st/ST_VID4025.html

My interpretation (grain of salt) is: they have “renewed” their CAPP/EAL4 certifications and certified (through documentation ONLY, no audits) their handling of security bulletins. This was, I think, the cornerstone of Shapiro’s assessment. Without code audits (and now, security process audits), this certification is useless.

However, I’m just a computer engineering student who follows your blog (which should be required reading — along with comp.risks — for us undergraduates in the computing field, but I digress), so take my comment on the current Windows CAPP/EAL4 + ALC_FLR.3 certification with a pillar of salt.

From my reading of FLR.3, I believe Microsoft has certified (heavily paraphrased and condensed):
– their internal ticketing tracks all reported problems for each release, and each problems’ nature, effect and status and resolution (patch, workaround, others). [FLR3.1, 3.2]
– users are able to read existing security bulletins and are notified of new security bulletins, if registered. [FLR3.3, 3.4, 3.9]
– a method for users to report or enquire about possible security problems is available. [FLR3.5, 3.8, 3.11]
– their internal ticketing system ensures reports aren’t lost or forgotten. [FLR3.3, 3.6, 3.9]
– there are safeguards in the development process to prevent introduction of new problems when correcting old ones. [FLR3.7]
– there’s a way for us to register so as to be notified of security bulletins. [FLR.3.10]

(I must be reading 3.5 and 3.8 wrong, because they both say the same, I think.)

Color me unimpressed. Microsoft has certified their “New Employee’s Guide to the Internal Ticketing System” manual.

Tyler Larson • December 21, 2005 12:46 AM

It’s worth pointing out that Windows is highly “securable.” Windows has an advanced access control subsystem with reasonably powerful ACLs which can be applied to nearly every resource that a program can access. It can transparently encrypt files using strong encryption technology, and has a host of other fairly advanced security features.

An appropriately configured Windows machine can be very secure. All major Microsoft applications are capable of running as a least privileged user, under which very little system damage can be done, even by malicous code.

That being said, nobody uses Windows in a secure configuration. Word might run as a least privileged user, but “Sally’s Recipe Finder” won’t. The ACL subsystem is powerful, but it is always bypassed through insecure configuration. Worms, viruses, spyware–they all rely on the assumption that you’ve effectively disabled Windows’ built-in security, which we all have.

Think of the Sony DRM Rootkit. Now, the word “Rootkit” comes from “root”, the administrator user under UNIX. In other words, you have to be administrator to use it. Think about it; the Sony rootkit worked by installing a KERNEL DRIVER. No OS is secure when the kernel can be modified at will.

Simon Johnosn • December 21, 2005 3:07 AM

Windows IS has half decent security properties but you have to set it up properly.

If this certification has any value, this is where it’s value lies.

If you use NTFS, don’t run as Administrator, run a virus checker and a hardware firewall (aswell as the inbuilt software one) and you only allow people to use a pre-approved list of software then Microsoft Windows has pretty good security properties.

I’ve been running in this kind of configuration with my parents, who like most parents are totally novices, for the last four years. The number of breakins? Zero!

For most people, computer security is about making your computer tough enough so that people don’t bother trying to break in to your machine and move on to a softer target.

The amount you have to raise the barrier to achieve this is surprisingly low.

Simon.

zara • December 21, 2005 5:38 AM

Yeah, tell me about it. I can’t seem to get any flavor of Linux running on my abacus! Sucks, I tell you!

There is quite a difference between something that doesn’t happen to have been ported to some platform, and a patch that makes the patched program stop running on a platform it was running on.

Just in case it wasn’t a light hearted jibe 😛

zara • December 21, 2005 5:40 AM

In other words, you have to be administrator to use it. Think about it; the Sony rootkit worked by installing a KERNEL DRIVER. No OS is secure when the kernel can be modified at will.

The vast majority of rootkits are installed after escalating privileges, they don’t just ask “please can you run me as root”.

Darren Moffat • December 21, 2005 5:42 AM

To Francois who said that on paper NT should be more secure than Unix like systems because it has “ACL’s, roles, administrators who aren’t all-powerful, better logging, etc”. Have you ever looked at Solaris ? Which has an EAL4+ with CAPP and RBACPP – NT’s roles aren’t included in their EAL4+ because it is only against CAPP.

Solaris has ACLs, roles, Audit all of which are included in the TOE and covered by the RBACPP. Whats more Solaris 10 is currently in evalution to CAPP and RBACPP. Solaris 10 will include Zones, and least privilege in the TOE. Our (currently in development) add on Trusted Extensions product for Solaris 10 will be evaluated with LSPP was well. Trusted Solaris 8 already has an EAL4+ CAPP, RBACPP, LSPP certificate. Sun has been doing this for a very long time, we did ITSEC EAL4+/E3 before CC even existed.

It is very important to note however for every operating system that is evaluated under CC it is a very specific configuration and unfortunately for most systems it isn’t the default config (this is true even of Solaris and Trusted Solaris).

Steve • December 21, 2005 5:52 PM

Funny how I see so much “anti MS” sentiment here with very little engineering thought… You don’t just “throw together” something and get it EAL 4 certified. Btw, NT 3.51 was what we used to call “Orange book” certified, not EAL certified. Orange book was trusted system, not trusted network which was “Red book”. So, by design, it wasn’t ever supposed to be C2 “Red book” when they never attempted to evaluate it under red book criteria. Having even a slight clue would help the anti-ms fools here.

Moving on, it also helps to understand the history of why CC even exists, or the CMM for that matter (capability maturity model). Back in the day, any idiot could make a paper company and write crappy software to sell to the government for big bucks. They would write the contracts such that very little could come back to haunt them if the customer (the Government) didn’t like what they got. CMM and the Rainbow books/EAL program all work together to make it harder to accept “crap” as software/hardware in a government procurement setting.

Linux, UNIX, Windows, etc. have the exact same problem space that they share equally. The only mitigating difference is market share and adoption rate, which therefore makes even a “small” Windows problem seem much worse than a critical UNIX kernel sploit.

Start focusing on the larger problems that have plagued IT for over 25 years and you’ll sooner realize Microsoft and a few other companies (Sun et. al.) are showing some leadership in this space.

Go consult the results at Openhack and tell me who got exploited and who didn’t…

Karl • December 22, 2005 8:50 AM

The claim that this the “the highest level granted to a commercial product” seems very strange. If nothing else, there is the XTS-400 running STOP that received EAL 5+ (under CAPP and LSPP):

http://www.digitalnet.com/solutions/information_assurance/xts400_trusted_sys.htm
http://niap.nist.gov/cc-scheme/st/ST_VID3012a.html

There are other commercial products in evaluation at much higher levels. For example, Integrity-178B is currently in evaluation for EAL 6+.

http://www.ghs.com/products/safety_critical/integrity-do-178b.html

KurtD • December 22, 2005 10:06 AM

Zara, you said “The vast majority of rootkits are installed after escalating privileges, they don’t just ask “please can you run me as root”.”

Where do you get this information? I’ve studied rootkits extensively and what I see over and over again is malware executed by niave users running as admin. No escalation of privileges necessary when the victim is running as admin. What escalation attacks are possible by an unprivileged user on a fully patched Windows system?

KurtD • December 22, 2005 10:13 AM

Dave, you state “What value is Microsoft’s LUA when it is FUBAR?” then point to a way to exploit a poorly implemented Software Restriction Policy (SRP). In order for Russinovich’s approach to work the user has to be able to run arbitrary code. If you are going to invest in deploying SRP you want to make sure users run with limited privileges and that they are unable to execute arbitrary code. That means that they should only have read & execute access to binaries that they are allowed to execute and that the SRP rules should block everything except the specifically allowed items, known as the ‘White List’ approach. You might want to make sure you actually understand how the exploit works and the underlying technology before slamming something in a public forum.

Ari Heikkinen • December 22, 2005 1:32 PM

Very few seem to get it that trying to fix security afterwards (when all the badly designed and implemented systems are already widely in use) is a very slow process, if not impossible. Bad design decisions that were made years ago can haunt for years or even decades. As for current mainstream operating systems, merely installing some patches and running a firewall aren’t going to solve the problem. The only way to fix the problem is to develop a new programming language with strictly defined syntax that will only make possible writing programs in a secure way and then rewrite everything including operating systems with it.

Anonymous Moi? • December 22, 2005 2:28 PM

@Karl
“Commercial” is probably being over-simplified here. EAL4 is typically the highest level for a general purpose (and not specifically designed as a security) product. It’s also a convenient level as that’s where “Mutual; Recognition” (certificates granted in one country being accepted by another) ends.

Anonymous • December 22, 2005 2:32 PM

@Tyler Larson,

“””Worms, viruses, spyware–they all rely on the assumption that you’ve effectively disabled Windows’ built-in security, which we all have.”””

I would add “[ … which we all] have because that’s the way it installs by default”

I think this is the most important point in the argument: It doesn’t matter much how secure you can make something by tweaking it because most people will use the defaults without changing them.

A much more meaningful security standard would be one that uses the system the way most people do (i.e. default install, or installed in whatever way the vendor ships it).

A standard based on a customised system assuming a benign environment seems utterly worthless, except as a marketing tool. It’s like claiming the hummvee is fuel efficient because it’ll roll downhill once you strip it down and put it on rails (assuming you have a tailwind).

Thomas Sprinkmeier • December 22, 2005 3:12 PM

I seem to be having an identity crisis this morning…..

prev post (re: defaults) was me.

Davi Ottenheimer • December 22, 2005 3:52 PM

“It doesn’t matter much how secure you can make something by tweaking it because most people will use the defaults without changing them”

An excellent point, but I seem to remember Microsoft marketing their software with security “features” and add-ons.

For example, you could either use NetMeeting or be secure. Pick one. Kind of like the MS philosophy has traditionally been you can either drive or wear a seatbelt, but not both — so the package that gets certified has seatbelts configured properly but isn’t driven anywhere.

Sven • December 24, 2005 3:32 AM

“so the package that gets certified has seatbelts configured properly but isn’t driven anywhere”

Excellent analogy, as the evaluated/certified WinXP does not have internet access (see link in my post above) 😉

Neville Wilkinson • January 18, 2006 4:38 AM

I don’t believe it is strictly accurate to say that the CC certification scheme is weak per se. I think it would be more accurate to say that EAL 4 does not signify a particularly high level of assurance. To achieve higher levels of assurance EAL 6/7 formal mathematical proof is required. Products at these levels are normally hardware based and include things like high-grade military cryptos and data diodes.

As was said earlier EAL4 is a largely paper based minimum standard for a general purpose OS. In practice it is intended to give Government system accreditors a warm feeling that the cleaners will find it difficult to log on to a classified network.

David von Oheimb • December 16, 2008 8:52 AM

EAL 4 is a relatively high assurance level, but most people do not realized that this is only one of many parameters of a CC certification. Equally important are the list of security objectives/threats, the boundries of the system to be evaluated (TOE), and the assumptions on the TOE environment. For a discussion on their effects on the value of a certificate, in particular the unrealistic assumptions made for the M$ Windows case, see http://en.wikipedia.org/wiki/Common_Criteria#Value_of_Certification

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.