Security Certifications

I’ve long been hostile to certifications—I’ve met too many bad security professionals with certifications and know many excellent security professionals without certifications. But, I’ve come to believe that, while certifications aren’t perfect, they’re a decent way for a security professional to learn some of the things he’s going to know, and a potential employer to assess whether a job candidate has the security expertise he’s going to need to know.

What’s changed? Both the job requirements and the certification programs.

Anyone can invent a security system that he himself cannot break. I’ve said this so often that Cory Doctorow has named it “Schneier’s Law”: When someone hands you a security system and says, “I believe this is secure,” the first thing you have to ask is, “Who the hell are you?” Show me what you’ve broken to demonstrate that your assertion of the system’s security means something.

That kind of expertise can’t be found in a certification. It’s a combination of an innate feel for security, extensive knowledge of the academic security literature, extensive experience in existing security systems, and practice. When I’ve hired people to design and evaluate security systems, I’ve paid no attention to certifications. They are meaningless; I need a different set of skills and abilities.

But most organizations don’t need to hire that kind of person. Network security has become standardized; organizations need a practitioner, not a researcher. This is good because there is so much demand for these practitioners that there aren’t enough researchers to go around. Certification programs are good at churning out practitioners.

And over the years, certification programs have gotten better. They really do teach knowledge that security practitioners need. I might not want a graduate designing a security protocol or evaluating a cryptosystem, but certs are fine for any of the handful of network security jobs a large organization needs.

At my company, we encourage our security analysts to take certification courses. We find that it’s the most cost-effective way to give them the skills they need to do ever-more-complex jobs.

Of course, none of this is perfect. I still meet bad security practitioners with certifications, and I still know excellent security professionals without any.

In the end, certifications are like profiling. They work , but they’re sloppy. Just because someone has a particular certification doesn’t mean that he has the security expertise you’re looking for (in other words, there are false positives). And just because someone doesn’t have a security certification doesn’t mean that he doesn’t have the required security expertise (false negatives). But we use them for the same reason we profile: We don’t have the time, patience, or ability to test for what we’re looking for explicitly.

Profiling based on security certifications is the easiest way for an organization to make a good hiring decision, and the easiest way for an organization to train its existing employees. And honestly, that’s usually good enough.

This essay originally appeared as a point-counterpoint with Marcus Ranum in the July 2006 issue of Information Security Magazine. (You have to fill out an annoying survey to read Marcus’s counterpoint, but 1) you can lie, and 2) it’s worth it.)

EDITED TO ADD (7/21): A Guide to Information Security Certifications.

EDITED TO ADD (9/11): Here’s Marcus’s column.

Tags: , , , , , ,

Posted on July 20, 2006 at 7:20 AM • 63 Comments

Comments

Michael Clark • July 20, 2006 7:50 AM

Do you have false negative and false positive reversed in the paragraph near the end? A certification without expertise is falsely saying something good, so a false positive.

arl • July 20, 2006 8:31 AM

To often I have heard people in different areas say “certification is meaningless” as an excuse as to why they don’t have one. My answer has always been that a certification helps establish a baseline for the process and is where evaluation begins. The most useful thing is that the certificaiton establishes that the holder “speaks the language” of the vendor or technology.

When I have had to deal with the volumes of people who are interested in a position I hear the “I don’t need certification” excuse all the time. For them my answer is simple, I have 200 people applying for this job and I don’t have time to quiz every one on their knowledge. Those with the requested certifications are invited in. That is where the screening process begins. And many of those with certifications won’t make the cut.

Maybe its like a drivers license, it does not tell me how good of a driver you are but at least we can get you behind the wheel and find out.

Daedala • July 20, 2006 8:38 AM

Certifications are what you make of them. When I was certified in Microsoft and looking for a job, I told interviewers point-blank that it was a paper certification that demonstrated my ability to learn, not my expertise. My security certifications are not paper certs, and I have the experience to prove I know what I’m doing. People can lie — but people lie on resumes all the time.

The CounterPoint, on the other hand, disturbs me a lot. The “old boy network” may work — if you want old boys who belong to a social elite regardless of merit. I doubt Ranum actually meant the sort of socially exclusive network most people mean by “old boy network.” Nevertheless, it’s a system that can mean women, minorities, and people with a lower-class background don’t get a fair chance. I was lucky enough to be able to name-drop on my resume and get an interview. Many others aren’t, and Ranum’s system won’t find them.

WhoNeedsCerts • July 20, 2006 9:06 AM

Here is something I never understood..

You are about to spend thousands and thousands of dollars, hours of effort, in bringing hopefully the right person in for the job.. You are about to lock yourself up into a position where you might have a tough time letting the person go if it faisl.. Yet you spend two minutes looking over a resume looking for key words or degrees to determine who you are going to bring in…

Give me the person who can detail (obviously not in book form) their work and relevant experience (in junior positions show me you can pick up technologies and be effective) over the cert loaded one everytime..

It’s ashame that the cover letter and resume are getting brushed over with keyword mania..

Mike Hamburg • July 20, 2006 9:19 AM

@pjm: While graduate degrees are similar in some ways, I don’t think they’re exactly the same, particularly Ph.Ds. While a certification means that you’ve completed some specific course or training, a Ph.D and in most cases a Master’s degree means you’ve done research.

They certify different skillsets, and so are appropriate for different jobs. You want a systems administrator to be certified, and you want a systems designer to have a Ph.D.

RvnPhnx • July 20, 2006 9:45 AM

Reminds me of GPAs: You can get a 4.0 without knowing much of anything, or you can know all the required information and still manage to get something in the 2-3 range (in other words, you did all of your work and learned the required information, you just didn’t do well on the test)–and anything in between.
The problem is that many HR people use GPA as the definitive test of the quality of the prospective employee. I’m stuck doing webservers for a living because I can’t cram for tests, but yet I was never asking “stupid” questions like “How’s an XOR gate work?” in the VLSI (chip design) class. Thankfully this often only applies to a college graduate’s first job–unfortunately it means that those of us with expensive technical degrees that didn’t pass the GPA-level of the week will almost never be able to get a job in the field we spent a large amount of money studying to work in.
Any one metric by itself is ALWAYS meaningless when dealing with people. This is just the way that it is being a human being.

@nonymou5 • July 20, 2006 9:54 AM

Here is some enlightening info from SANS NewsBites e-mails.

The Editorial Board of SANS NewsBites

…

Bruce Schneier has authored eight books — including BEYOND FEAR and
SECRETS AND LIES — and dozens of articles and academic papers.
Schneier has regularly appeared on television and radio, has testified
before Congress, and is a frequent writer and lecturer on issues
surrounding security and privacy.

…

Marcus J. Ranum built the first firewall for the White House and is
widely recognized as a security products designer and industry
innovator.

So Bruce and Marcus should I consider this an endorsement of SANS certification?

SaveLebanon • July 20, 2006 10:17 AM

Please go to http://julywar.epetition.net and sign the Save the Lebanese Civilians Petition and forward this invitation to your friends. If you think this has nothing to do with security, think again! Israeli bombs, paid by the US taxpayer, are not only killing hundreds of civilians but are also recruiting a whole new generation of terorists.

Lebanese civilians have been under the constant attack of the state of Israel for several days. The State of Israel, in disregard to international law and the Geneva Convention, is launching a maritime and air siege targeting the entire population of the country. Innocent civilians are being collectively punished in Lebanon by the state of Israel in deliberate acts of terrorism as described in Article 33 of the Geneva Convention.

This is from Juan Cole (http://www.juancole.com/2006/07/israel-targets-milk-medicine-factories.html):

“Only for those with really strong stomachs. This is what some of the hundreds of civilians killed by the Israeli military in Lebanon look like. Very graphic and disturbing. I disavow the labeling in the site. But this is a war, and this is what war looks like, and I think it is necessary to stare it right in the face:”
*** http://fromisrael2lebanon.com/

Anonymous • July 20, 2006 10:22 AM

@Marcus Ranum

>For example, if someone wanted to hire me to lock down an ULTRIX 3.1d system, I’m eminently qualified. But I’d be at a loss when presented with today’s confusing plethora of Linux “distros”–I’d need months of studying and experimenting before I’d be ready to work on one of them. But if I had a certification, maybe someone would hire me by mistake, thinking I was qualified, and then I could do that retraining on the company’s nickel. If someone asked one of my peers who they’d recommend for a Linux project, I’m sure my name wouldn’t come up. But if the job called for a “senior curmudgeon,” well, that would be another story entirely.

So you would not consider someone who had a SANS Gold level certification to lockdown a Linux machine?

SANS Unix/Linux Cert
http://www.sans.org/training/description.php?tid=245

What is the GIAC GOLD level?
http://www.giac.org/gold/

Also since you have taugh SANS classses for both Securing LAMP and for Log Analysis I would think you would have endorsed SANS certifications.

Peter Pearson • July 20, 2006 10:29 AM

Is the emergence of certification programs perhaps an indication that the exciting times are over, and it’s time to move on to another field? The relative importance of “figuring things out” versus “memorizing many standards” has slid decidedly in favor of the latter.

Lynnski • July 20, 2006 10:32 AM

Sure, that works fine if your race, gender, ethnicity and age are such that people *expect * you to be expert in security. But if you’re not a middle-aged white Jewish guy, then people are not so predisposed to think you know what you’re talking about. Certifications are a baseline, and they may provide an avenue for equity and access in a field where these qualities are glaringly absent.

derf • July 20, 2006 11:03 AM

When someone hands you a security system and says, “I believe this is secure,” the first thing you have to ask is, “Define ‘secure’.”

There is no 100% security, so what do they mean when they say secure? Secure against what? Is it secure against 14 year olds without dates? Is it secure against a Category 5 hurricane? Is it secure against malicious insiders? How long is this supposed secure-ness good for?

There isn’t a certification in the world that can duplicate the experience of having been in the trenches. I know people in the industry with more certs falling from their bung than I even knew existed, but I can’t put them on a single job alone, because they don’t know how to DO anything except take tests.

@nonymou5 • July 20, 2006 11:17 AM

Another thought on certification is they are not all equal.

There are Vendor Certs.
Microsoft’s MCP/MCSE, CISCO CCNA/CCNP/CCIE
Pro: The canidate is likley to know how to work on your specific platform.
Con: The canidate is likely to think in only the vendor’s interest.

There are Certs to assure knowledge of standard security terminlogy.
ISC CISSP
Pro: Can talk strategy and evaluate the nine domains to evaluate how the company is doing overall
Cons: Most likely could not tell you what the nineth byte of an ip packet means or if OpenSSL is out of date on Red Hat Linux.

Topic specific, vendor neutral.
SANS GIAC
Pro: Vendor neutral. A lot of focus on specific skills in NIDS or Hardening Windows, Incident Handeling, etc.
Con: Concentration on open source tools since they are easily available, but it does not seem to impress all employers.

Pat Cahalan • July 20, 2006 11:36 AM

grumble

Filled out the form, and every time I try to get to the article it asks me to fill out the form again.

On topic:
Certifications (and educational programs for that matter) are generally a good thing. Of course you shouldn’t rely on “does this guy have an MCSE” or “does this guy have his CCNA”, just like you shouldn’t rely on “does he have an BS degree” or “does he have 10 years experience” (I’ll admit, HR departments do, but that’s neither here nor there).

People learn in different ways. You learn from academic programs (teaching theory), you learn from certifications (teaching practice), you learn from on the job experience (seeing how practice and theory interact with realities like budgeting and manpower), and you learn from drinking beer with guys like Marcus or Bruce at a bar during a conference (what I affectionately call brain-sucking).

Saying that any one of these methods of learning is bad is wrong, just like saying one of these methods is “the best” is wrong. Experience is not always the best teacher – I know plenty of people who have been in the IT business for 5, 10, or more years who have been doing the same thing the same way for a long time… and that way isn’t the safest, or most efficient, or most easily scalable, or cost effective… it’s just the way that they’ve been doing it for 10 years. It might have been the best way to do things while they were working for Boeing for 8 years, but that doesn’t mean it is the best way to do things at the small startup they’re working for now, or the consulting business they want to start on their own, etc.

And if Marcus pokes his head in the thread -> you’re correct in the sense that if someone hires you just because you have a piece of paper that lets you tack a bunch of letters on the end of your resume they may not be getting what they think they’re getting. One drawback of certifications is that there are so many of them, and unless you know something about the certifications the fact that people have them doesn’t tell you anything other than the fact that they were willing to plunk down somewhere between $50 and $4000 and spend somewhere between 1/2 day (say, at a web seminar) and a year and a half of dedicated studying to pass an exam.

Getting your CCNE or your CISSP certainly does indicate something compelling.

Tangent – I dispute that it would take you months to figure out how to “lock down” a linux box (or a windows box, or an Amiga for that matter). You’ve been doing security long enough that you know how to do a simple analysis of threat. You can scan the machine, figure out what ports it is listening to, turn off services that aren’t necessary, and read up on what needs to be done to secure the remaining network-facing services probably in a day, maybe a few if the box is really complicated. It might take you a month to really lock it down by stripping out all the unnecessary applications, making sure the remaining services are running relatively securely, etc. It might take you a year to figure out how to really lock it down by hacking down to the kernel to remove the possibility of local exploits and physically securing the box in every way, shape, and form you can think of… but everything past that first week is going to be limited and delayed by budget, interaction with other systems, political reasons, physical location, etc… all of which are going to affect you whether you’re Linus Torvalds or Marcus Ranum or Mark Russinovich or Bruce Schneier, and regardless of your level of technical syntax expertise in the particular system you’re trying to lock down.

I’d rather hire someone who’s been doing security for 10 years regardless of the system than hire someone who has been doing Linux kernel hacking for 10 years but not doing anything to expand her knowledge of security to be a security person… even if the security is going to be all-Linux 🙂

Mike Sherwood • July 20, 2006 11:41 AM

Certifications are a way to ensure that someone has at least a minimum level of exposure to a field. This may be helpful when staffing less experienced positions. The certification represents a commitment to a particular field. The high cost of training/study materials/tests makes certification an expensive process for people trying to get into a field.

I recently got certified as a Sun System and Network administrator at the request of an employer who paid for it all. Reading a book on the topics being tested was all I needed to pass all three exams on the first try. The experience demonstrated to me that my lack of respect for the certification process is well founded. The book I used was very poorly edited, resulting in giving misleading or wrong information in some cases. There were some questions on the test that seemed odd. I just couldn’t see how the information would be useful enough in the real world to ensure that everyone in the field knows about it, while completely ignoring other topics that I would expect to be much more important.

I cringe when people use certifications as a baseline for determining who should be interviewed. I see this with HR people who need to ensure that everyone’s papers are in order. I think the certifications tend to give a false sense of confidence to those doing the hiring when they lack the ability to determine if a candidate is able to perform the job.

Gee Whiz • July 20, 2006 11:45 AM

It is important for us all to realize that certifications are merely “low water marks, not high water marks. For high water marks, we have to look at the experience and accomplishments of a given candidate but even this isn’t fool proof. For example, some of the dumbest people I know are PhD’s.

While I used to think that companies should hire the best and the brightest, I have found that there is no substitute for diligence. Diligence without intelligence might get you somewhere but intelligence without diligence never succeeds.

Anonymous • July 20, 2006 11:57 AM

@bruce:

“Who the hell are you?” Show me what you’ve broken to demonstrate that your assertion of the system’s security means something.

But I’d like to note that someone who can break into things doesn’t necessarily design good security products.
For example, if you hire a successful burglar who broke the doors for a secure house design and he installs reinforced doors, what about the windows, the chimney, the walls…
If someone knows how to break (into) stuff this doesn’t implicate he knows how to effectively protect the same stuff against breaking.
Security is a process, breaking it requires just knowledge about one “weak link”.

Regards,
Roy

Stefan Wagner • July 20, 2006 12:13 PM

My experience with Java-Certifications is:
If a person mentions a lot of them in a forum-signature, he will ask a very stupid question by 90% probability.

Often questions a beginner can simply find out by trying.

They often can answer questions which aren’t asked in real life, and often don’t know how to use their theoretical knowledge practical.

The certifications suffer from questions, which fit well into a testing scheme (Yes/No., a/b/c) and might be evaluated automatically.
They test, what is cheap to test and fits to the theme.

I would expect sec-certs to be similar.

Yvan • July 20, 2006 1:55 PM

Certifications are always interesting, and their growing importance seems to be indicitive of the commodization of security, which, overall is a good thing.

If everyone is competing to sell firewalls(s/firewall/general security technology) based on value-added features, then it means that firewalls will be cheaper, higher quality (in theory), and easier to use, which results in a net increase in the adoption of firewalls. An increase in the adoption of firewalls means that more organizations are attempting to improve security, and as the saying goes, the first step is admitting you have a problem :).

This type of commodization is broadly observed in many fields such as engineering, medicine, education, and frequently results in the formation of trade groups and professional organizations. Again, this is a good thing.

When strong professional organizations emerge they can dictate minimum standards, either through the form of unions or professional organizations. Once this begins to happen, we will see an improvement in the quality of security practicioners as it becomes more acceptable for security practicioners to be a Firewall specialist, or an IDS Specialist, or a Content-Filtering specialist, since trade groups will naturally create spaces for these specialists to accomodate the needs of the community they support.

The current state of security certification tends to focus on the Jack-of-all-Trades approach to security, i.e. CISSP CBK domains, which is deeply flawed; in reality most people lack the technical expertise, or the passion for the field, to actually become security researchers. This follows other fields; In engineering you have a wide range of trade specialists who support both the engineers and the businesses built on engineered products (mechanics, plumbers, electricians, etc), medicine is very similar; there are an army of nurses, lab technicians, pharmacy technicians, etc that support each hospital, and without which, most doctors would not be able to function.

The growing adoption of these certifications will allow these types of trades specialists to emerge, and will lead to stabilization in certain aspects of the security space. The only challenge is that we are at a cusp; the bar to gain entry into the field has been lowered, but the metrics used are highly suspect among people who are more experienced who forged the path by the people who are following. People like Bruce Schneier, Marcus Ranum, Marty Roesch, Renaud Desraison, (the list could go on and on) cleared the way by helping, each in their own way, with this commodization, and it makes sense that overall, they would be leery of the people sauntering down the path they just cleared, especially the ones who are wandering around claiming they should be there because they passed a test.

The road has been cleared, and like every new trail, the first people running down the new path are those hoping to gain the most from getting there before others, for some this means money, and for some (at least, me) it means the opportunity to learn more about an extremely fascinating field of study.

Chris • July 20, 2006 3:09 PM

Reading this thread, an old quote comes to mind that used to be popular in many IT departments: “Nobody ever got fired for buying IBM.”

Hiring a person with the right letters after their name provides some security for the person doing the hiring: they have a ready excuse if things don’t work out. He was certified in XXX! While this defense may not hold up to serious scrutiny, I have personally seen it used successfully to save an interviewer’s posterior when a new hire was terminated only 3 weeks after being hired. Looking for a certification is partly just a hedge against a bet on hiring someone.

Certifications also provide a (flawed) mechanism for communicating with the people, untrained in your field, who will perform your candidate search. These people often do not understand the terminology and shorthand of the industry. Any description of the skillset you’re looking for will sound nonsensical: “I need someone who can dogfood the framework of a glass pasture.” Sounds like Total Jibberish. I know what it means but to an HR person it’s a worthless set of criteria. If you tell them you need someone with a CISSP certification then they have something they can search on.

WhoNeedsCerts • July 20, 2006 3:52 PM

@Chris:
“He was certified in XXX!”

Umm.. what industry do you work in!!! 😉

Ralph • July 20, 2006 6:08 PM

To date I have been against academic qualifications in our field, few of the most important lessons that help you at the coal face can be taught. We teach on the job.

However, I have become increasingly aware that most security people I work with, in end user land, have MAJOR holes in their basic methodology.

It’s as if they don’t understand the context they are working in, how to prioritise and use good diagnostic process with discipline. At times they are downright superstitious.

This in turn places more stress back onto the few who know what they are doing. It also results in a lot of wasted money and security that is in name only.

As our industry grows we must address this and maybe the time has come to cover security basics in acadamia before you step into the front lines.

Of all the “weak links” in the security chain, it is the people ones I worry about most.

Justin • July 20, 2006 6:49 PM

What’s the difference between having a certification on your resume and listing ‘5 years experience’? In either case, they can be lies, they can be true but misleading because the person’s an idiot, or they can correlate well with the person’s actual abilities. In both cases you’re taking an identical risk when you either retain or toss that resume on the basis of that fact.

CPCcurmudgeon • July 20, 2006 8:01 PM

Interesting discussion that comes up in lots of forums frequented by people with certs (or skills that may require certs), such as systems and network administration. I have mixed feelings on the subject. On one hand, certifications can give an employer a baseline upon which to judge a candidate. OTOH, what the certification tests may not prove to be the defining factor in whether or not someone can do a job.

Some industries for which certs are required such as medicine generally have success (that is, on average, the clients are satisfied). Yet the jury is still out on whether certs will prove to be reliable predictors of success in hiring.

I don’t know much about the history of medicine, but I wonder if after certs were introduced, there was a period of transition in which some people who, despite having success in servicing people in need of medical attention, became displaced — they experienced difficulty finding jobs once certs were required. I think part of the problem is that most of the “researcher” types fall into that category — for most of their careers, they have been taught to take an academic approach to their work, but are suddenly required to memorize large quantities of information. (I have experienced this problem at interviews, for example; even though certs aren’t required, the interview questions often require detailed knowledge which when doing a job, someone can easily reference in man pages or on the web.)

Davi Ottenheimer • July 20, 2006 10:59 PM

This debate is probably as old as the term “professional”.

“Show me what you’ve broken to demonstrate that your assertion of the system’s security means something.”

The danger of that kind of strongly worded statement is that a value system is implied when “breaking” something.

Are you talking about the curious sort of person that might reverse-engineer an engine (by breaking it down) just to understand it and then put it together so that it runs even better than before, or are you talking about the guy that thinks that stealing is really a good way to make a living especially if they can sell the break-in secrets (that they probably learned from someone else) as well as keep the loot?

Ian Woollard • July 21, 2006 12:09 AM

@Bruce, is this a roundabout way to announce a new certification program of yours? :p

Ogundaini Segun • July 21, 2006 12:15 AM

Some certifications are LABs where real world scenarios is tested. E.g CCIE, RHCE.

Also we need to define what we call experience. 10 years experience most of the time might be 1 year experience multiply by 10.

erasmus • July 21, 2006 2:49 AM

Remember how Strawman became clever when the Wizard of Oz gave him a diploma?

Its useful to understand the motivation of someone who has gone down the certification route – was it because of professionalism*, opportunism, lack of confidence, trying to prove something, etc…?

*In the older professions, certification is a serious business that takes 5-10 years, inculcates a series of attitudes, and involves personal liability for accidents and mistakes.

averros • July 21, 2006 4:14 AM

When I see someone mentioning his certifications, my first thought is “this guy had too much spare time on his hands”.

Meaning he’s no good for the actual work: as we all know being good guarantees that more and more work will be coming your way.

Anon-CISSP • July 21, 2006 10:18 AM

I agree that certs are basically worthless. I have the CISSP (my employer’s decision) and I’ve found it offers me really no value other than something to put on a resume. Since I’ve got a nice job now, all the CISSP gives me is the hassle of submitting the yearly fee (~100/yr) to accounts payable for payment and keeping up on my “educational credits.” Not to mention that when I took the CISSP that’s all there was from ISC2 — now they’re dilluting it with the SSCP and extra concentrations like “certified engineer” and others to try to milk the cash cow even more. It’s absolutely pathetic.

Look at the CISM, they basically grandfathered anyone who wanted it a few years ago to build a base of certified people to springboard the cert into the market. A certification should provide assurance that the certified individual meets a specific standard — what about the grandfathered CISMs? The assurance with them is that they had $500 and could fill out the application for the certificate.

I don’t think this discussion is limited to certifications — a lot of degree programs are going this route too. Schools that were considered “trade schools” a few years ago have all seemed to become accredited and are pumping out bachelor’s degrees in things like “Computer Science TECHNOLOGY,” or “Information Systems Security.” The ones that do post the curriculum for these programs on the Web show classes that are light in mathematics and science, and heavy in job-oriented classes — like “Windows Web Server Configuration.” As a hiring manager who has a (real) Computer Science undergraduate degree, I wouldn’t even consider hiring someone with those types of degrees. And don’t get me started about the SANS “Master of Science” program (sans.edu) — attend 6 conferences and get a master’s — give me a break…..

soliare • July 21, 2006 10:20 AM

“Filled out the form, and every time I try to get to the article it asks me to fill out the form again.”

Ditto. Something’s really broken, maybe only for non-Windoze people.

I too have mixed feelings about certs. I don’t have any, but I can see their utility for lazy HR people, just like university degrees (my BSCS degree might as easily have been in computer philosophy – when was the last time you needed to know complexity classes in a real program?).

Really, hiring the right people is so absolutely critical to the success of your business that you can’t afford not to test them yourself. I’ll take a sharp guy with no experience over a dull guy with lots of experience, assuming that the sharp guy can gather up most of the information by reading some of the key books or web sites. I personally don’t have a good memory, but I have excellent search skills, and I can remember enough to find it in a very short period of time. Certifications are expensive and expire to quickly — they’re really not about serving the industry, they’re about serving the company (or organization) collecting money for the certs. The non-profits or non-corporate cert orgs are battling for reputation of their org and perceived value of their certs.

If an organization doesn’t have the expertise to test someone or identify some expert to give them the once-over, then IT and/or security is probably not their core competency, and you will probably be unappreciated and seen as a cost-center (especially in security). Maybe that’s okay. Maybe your skills really are so-so, and getting a cert is proof that you can fill the security needs of the average law firm or doctor’s office. Fine.

Personally, I can tell just by talking to people, by listening to the way they express themselves and the wording they use, how competent they are (in most cases). The same thing applies to the written word; I can recognize genius when I read Aristotle or Newton or anyone of that caliber. I have far more respect (and am thrilled) by companies that actually have some kind of test for prospective employees.

And anyway, hiring decisions are not the end of the testing process. Your first year, your second year, your whole career you are evaluated. The chaff will get pink slips during bust cycles, and by aggressively seeking the best employees, rewarding exceptional talent, constructive criticism, and performance monitoring, your enterprise will be much better off.

The programming industry is one of the very few industries where the best outperform the worst by two orders of magnitude or more. With security, there’s no lines-of-code metrics (however flawed those are), but it’s clear that the best security folks are going to prevent intrusions, data leaks, data loss, downtime, and other resource-intensive problems far better than the average joe. They will also catch problems earlier on, instead of not at all. I’m not sure you can put a dollar value on having your customer database pilfered, or on knowing that it was pilfered. Can you quantify how much theft of your social security number and name/address is worth? As computers handle more and more data, this will become more and more important. There are actually hospitals with FDA-certified general-purpose computers that run e.g. MRIs, and hospital staff who plug them in to the Internet because they want to check their email. Need I mention that these Windoze machines cannot be patched because the FDA does not allow you to modify them after they are certified without losing that certification, and that they aren’t normally on the Internet where they could download patches anyway? How much is an iron lung worth to you, should you or your loved one need one? How about voting systems with wireless cards in them?

Dr. Neal Krawetz • July 21, 2006 10:55 AM

I discuss a similar viewpoint in my book, “Introduction to Network Security”.

Certifications are a great idea, but they are currently implemented poorly. Some of the failures include:

  • Uneven experience. Many certs allow the substitution of college courses or degrees in place of required work experience. The problem is, not all schools are the same. A CS degree from Harvard that focuses on Graphics is not the same as a “How to use Wordperfect” class from Dropdown College or a computer security course from Stanford. While colleges weigh transfers courses, most certs do not — all courses weigh the same.
  • Uneven continuing education. A talk at Blackhat will give you CPE credits. The exact same talk two days later at Defcon will not. This suggests that CPE credits depend on the forum and not the content.
  • Making matters worse: you can buy CPE credits. Paying for the course is not the same as attending. And some groups (e.g., ISSA) give CPE credits to non-security positions like regional secretary.
  • There is no common baseline for certifications. Anyone with a “Bachelors” in CS has a baseline level of competancy. Yet GIAC and CISSP and Security+ are not comparable certs (like comparing apples to onions — they look similar from a high level, but taste very different).

  • Some certs have ethical requirements. Yet they have no method of enforcing ethics. Some ethical requirements (CISSP) conflict. (I like the “treat all constituents fairly” and “give precidence to other CISSPs”. Double standard?)

While the medical and legal professions have Bar associations for enforcement and complaint management, this is missing from the computer security profession.

Until there is a unified standard, certs are a good idea in concept but not worth much more than a boyscout badge. (No offense to the boyscouts.)

Bryan Feir • July 21, 2006 11:11 AM

@erasmus:
“Remember how Strawman became clever when the Wizard of Oz gave him a diploma?”

Which, of course, was more an issue of self-confidence than anything else; the diploma had absolutely no effect on his actual intelligence. (Part of the whole point of that Wizard of Oz scene was that all four members of the party already had what they were searching for, they just didn’t know it.)

Really, the main thing any certification proves is that you were willing to go through the effort to get it. I’ve had that quoted to me explicitly as the reason some people look for University degrees: it doesn’t necessarily mean you’re intelligent, but it does mean you were willing to spend four years working toward that degree.

And with some certifications, this doesn’t prove anything as there’s not much effort required to get them.

Corey • July 21, 2006 11:47 AM

Certs are best for trying to circumvent a lately-growing problem: Because of corporate short-sighted behavior, nobody want to train anybody. Therefore, it’s very difficult to get a job unless you’ve done the exact same job before. Some HR drones will grudgingly accept a cert in lieu of having done the exact same job before.

Some certs require work experience, which does increase their value (it makes a more positive statement about the person’s skills). However, they then lose their usefulness for breaking in to a new job.

For example, I looked into studying and taking ASE certification tests in an attempt to circumvent the India Threat. They require experience, though, so they’re useless for breaking into the auto mechanicry industry.

Anon-CISSP • July 21, 2006 12:28 PM

@soliare

Great points all around. I too find things like “Analysis of Algorithms” something I don’t call upon very often. I find myself using math (mostly combinatorics, stats, and probability) more than most of the CS curriculum.

On another note “Information Security” magazine has changed quite a bit since ICSA sold them, now they seem to be in the business of tooting the horns of the “Top Security People” in the industry, and the latest edition is no exception. Looking through the bios of these CISOs, it appears that most do not hold CS/Math/Engineering degrees. This, is partially the cause of why certifications are so popular. Companies are placing people with no understanding of technology into technology leadership roles. Sure, a CISO shouldn’t be hands on installing a firewall, but it would be nice if they understood risk as it applies to computer architecture. I’ve never seen a CFO without a strong financial background or a COO without a strong background in ops, but for some reasons companies think it’s okay to drop someone with a degree in Art History into a CISO role. What winds up happening is weak CISOs are held at the mercy of their top technical people and you wind up having prima donnas within the department.

Case in point, I recently attended a sales presentation by one of the companies that is selling products that protect you against ALL* application overflows. The sales engineer started rambling about the stack and instruction pointers in an attempt to confuse everyone. One of my peers, a manager in charge of Windows systems asked “If the stack is so dangerous, why do computers use it? Can’t they use something else?” It wouldn’t have been so embarrassing if the four other managers and director in the room didn’t act like it was a great question.

This is as ludicrous as telling the CFO to multiply profits by 10, and divide costs by 10. Since 10/10 = 1, you’re basically multiplying everything by 1 so it’s okay. This is equivalent to what CISOs and other IT people are falling for, hook, line, and sinker.

*ALL later defined by them as any attacks they have a signature for.

Pat Cahalan • July 21, 2006 12:51 PM

There is one additional factor.

Certifications and degrees do tell hiring managers, hr people, and even technical managers that the candidate in question is actively trying to make him/herself a more attractive candidate.

Someone who has 10 years experience, has a smattering of certifications and a bachelor’s degree in electrical engineering, software engineerig, computer science, or mathematics has put forth some serious effort. A master’s degree indicates more effort.

I’ll grant you, in some cases, the actual technical knowledge gained may be marginal, but discounting certification or academic learning because there are some bad programs out there is throwing the baby out with the bathwater. Finding out the rating of an academic institution is pretty simple.

Getting through a bunch of certifications requires either shelling out your own money or convincing your previous employer to foot the bill. The first case shows some self sacrifice for future good, the second shows at least a marginal level of political moxie. Getting through a degree program shows the same, and for the most part requires that you execute certain deliverables on time and under pressure. Getting through an advanced degree program again shows commitment, and indicates an ability to do collaborate and/or generate original material.

Sure, there are bad degree programs. There are good degree programs that nevertheless teach habits and skills that may not be applicable for the particular position you’re trying to fill, as a hiring manager. There are certifications that are easily obtained.

But someone who has experience, degree(s), and certification(s) is more likely to be a more effective employee than someone who has only one or two of the above.

@nonymou5 • July 21, 2006 2:37 PM

What really cracks me up about all the elitism being thrown around here is that many of the same people will also state that no one tries to learn about security.

A holder of a cert may not design protocols, but it shows the person took effort to learn some aspect of security. A person may learn the same stuff on his/her own, but that is not easy to reflect on a resume. I decided a few years ago that whatever I learn to expand my own knowledge, so also be accompanied with some record/letters.

What I am seeing is that same people who tell others to learn security will also discourage them from doing so (That is unless give up your day job and go to a four year university).

It would be really nice to get beyond the stories about how someone with a cert stunk and what have the certs done right?

Here are some items I think they have done right.

Vendor Certs: Establish a training baseline that this is how we the vendor expects the product to work.

CISSP: Establish a common framework and language so that people with different jobs, companies, degrees, can at least talk the same language and think in a common framework about how to approach security.

SANS GIAC: Learn a baseline of specific skills for specific topics such as Incident handling. I doubt many of your CS degrees taught you anything about dealing with collection of evidence (Chain of custody) and working with the FBI. For those who got their CS degree 15 years ago, I doubt you learned about snort and if you did so on your own, how do you show that on your resume?

So let’s hear some specifics of what would you do to make certs up to your standards.

Anon-CISSP • July 21, 2006 3:42 PM

@ @nonymou5

Sure, someone with absolutely no experience or degree might do well with a certification. It’s definitely better than nothing when trying for an entry-level job. Typically one shows skills on their resume with related work experience. I’ve got references to Python on my resume, but no certifications to “back it up.” Doesn’t mean I don’t know it.

Vendor certifications are great, if I managed a group that dealt with firewalls, I’d want everybody in that group to have the highest vendor certification for the firewall they use. It shows they know the product, which essentially is what they’re paid to do.

As far as I know, the ISC2 didn’t create any of the language or frameworks it tests on. Most of that was developed by IEEE, ACM, NIST, ANSI, ISO, and academia.

I’ve attended one SANS conference on Forensics and wasn’t impressed — the instructor kept pronouncing Mac OS X as Mac OS “ecks,” not as Mac OS “ten.” He also couldn’t pronounce Ethereal properly. The instructors I’ve met there all basically came off as a group of amateurs who learned one aspect of security very well and would rub your nose in it any chance they got. You’d probably be better off joining Infraguard and networking with real FBI representatives as opposed to having SANS tell you the same thing for $2000.

Finally CS isn’t about learning how to use applications, it’s about using systems to solve problems efficiently. You don’t need a CS degree to install and use Snort, though you’d probably need one if you wanted to develop a mathematical model of anomalous activity with the data that Snort produced.

Pat Cahalan • July 21, 2006 4:13 PM

CS isn’t about learning how to use applications, it’s about using systems
to solve problems efficiently.

Slightly pedantic, but I don’t think this is generally accurate.

I think Computer Science (in most academic organizations) is concered with theory and new knowledge horizons. Tools are built to test hypothesis. Efficiency is an element, but not a primary focus. You build to learn or test, not to do.

Compare this to general Engineering which is concerned with building things for a particular deployment purpose, where efficiency is a primary focus.

This is why a great many brilliant computer scientists aren’t necessarily great software development team members, at least right out of school.

Anonymous • July 21, 2006 5:27 PM

@Anon-CISSP

Typically one shows skills on their resume with related work experience.

And when you are trying to get in that specific line of work experience, should I just lie or should I just keep hoping someone will see my honest intentions.

I’ve got references to Python on my resume, but no certifications to “back it up.” Doesn’t mean I don’t know it..

I was not arguing that you would not know Python. But you have experience from work that you can reference on your resume. Now imagine if your work would not let you use Python on your job. It would be easy to show python experience since you could always bring a CD of your scripts. Now how about running snort. That’s not as easy to bring a CD of scripts nor since your workl did not have you as a NIDS admin you can’t honestly claim that. Hence certification is the next choice.

I have learned a lot of things on my own, but when I realized it was hard to qualify them on a resume, that’s when it made sense get cert letters.

As far as I know, the ISC2 didn’t create any of the language or frameworks it tests on.
I used the word establish, maybe I should use the word commoditized. Anyway I did not attempt to say they alone created the framework. But if you want to learn to speak a common security lanaguge in a short amount of time, the CISSP is a good option.

I’ve attended one SANS conference

I also attened University of California were some professors sucked at teaching. So what’s your point?

SANS has also had the following as instructors Marcus Ranum, Marty Rosch, Tina Byrd, Chris Brenton, would you say these folks also suck?

Vendor certifications are great, if I managed a group that dealt with firewalls, I’d want everybody in that group to have the highest vendor certification for the firewall they use. It shows they know the product, which essentially is what they’re paid to do.

So to make sure I am understanding you, Vendor Certs: OK. Vendor neutral Certs, not OK.

You don’t need a CS degree to install and use Snort…

To install and use snort, no. But without a background in reading packets, it isn’t that useful. If you don’t have a background in reading packets then how will you chase down the possibility of that false positive? Hence a CS degree person may have an easier time with snort the just Joe user who installed snort and ran it on his network. But if Joe user wanted to learn how to read packets and chase down the false positives, would you want him to go back for more college when the SANS GCIA class will do that for him?

Anon-CISSP • July 21, 2006 8:11 PM

@Anonymous

Hence certification is the next choice.

So in other words, you have no production experience to document on your resume, but are using the certification to show that you know enough to pass a test? Isn’t that what this whole thread is about? That certs basically just show who is capable of passing a test and is not an indicator of experience?

But if you want to learn to speak a common security lanaguge in a short amount of time, the CISSP is a good option.

Doesn’t the CISSP requires that you have 4 years experience in Information Security before even taking the exam? If you can’t get the language of security down in four years you have no business working in this field.

I also attened University of California were some professors sucked at teaching. So what’s your point?

The point is for the amount of money SANS costs, you should get an instructor who has the capability to articulate the technology using the correct pronunciation.

SANS has also had the following as instructors Marcus Ranum, Marty Rosch, Tina Byrd, Chris Brenton, would you say these folks also suck?

I know the first two — never heard of the other two. They must have made their name on the security lecture circuit when I was busy working on algorithms at Bell Labs.

So to make sure I am understanding you, Vendor Certs: OK. Vendor neutral Certs, not OK.

I never said that. If I managed a group of firewall admins I’d want them to have the certificate, but I wouldn’t blindly hire them based only on the certification. Read the article quoted by Bruce — Marcus takes the same position I do on this one. Go and attack him.

But if Joe user wanted to learn how to read packets and chase down the false positives, would you want him to go back for more college when the SANS GCIA class will do that for him?

I never said that a CS degree was the ultimate credential in security — like the CISSP I’ve met a lot of people who went through college doing the bare minimum to get by.

Anon-CISSP • July 21, 2006 8:16 PM

@Pat Cahalan

Very true — if I had wanted to be a programmer/developer I would have probably got a degree in Software Engineering and not CS.

Anonymous • July 21, 2006 10:06 PM

@ Anon-CISSP
Tina Byrd, member of the schmoo group.
Collaborator in research with Marcus Ranum on log analysis.
http://www.loganalysis.org/

Chris Brenton
Author of the following books:
http://www.amazon.com/s/102-1628516-2098511?ie=UTF8&index=books&rank=-relevance%2C%2Bavailability%2C-daterank&field-author-exact=Chris%20Brenton

They must have made their name on the security lecture circuit when I was busy working on algorithms at Bell Labs.

Wow. You sure are impressive. I guess you have nothing to learn from these folks. I guess you expect the guy who wants to manage the network at the local hospital to design algorithms? Oh wait, that would not be in his job description. Because that would be a diferent job.

That certs basically just show who is capable of passing a test and is not an indicator of experience?

That’s where it depends on the cert.
Some certs require just a test CISSP. Other certs require a good level of effort SANS GSE. Will these people be desiging algorithms for Bell Labs, no. Can it be a factor for the guy who wants to run your nids for your network, yes.

Marcus takes the same position I do on this one

This is one area where I disagree with Marcus. I admire him in many ways, but his talent and in some ways luck, has baised his view to the rest the world. The funny part is Marcus is always complaining that people are bind to security. Yet one purpose of a Cert is to help teach enough security to overcome this blindness.

I never said that a CS degree was the ultimate credential in security — like the CISSP I’ve met a lot of people who went through college doing the bare minimum to get by.

Then hence what’s your problem with certs? It shows someone is putting effort into learning and desires experience to improve himself and career. It’s not like mom and dad pushed the person to get the certs. But I guess if you can’t design algorithms at Bell Labs you should jsut go sell insurance.

So to make sure I am understanding you, Vendor Certs: OK. Vendor neutral Certs, not OK.

I never said that.

What you said was you would want your group to get certs for your vendors products. But at the same time you have blasted the vendor netural certs. What other conclusion can I draw? Do you see anything redeemable in vendor neutral certs? When you went to the SANS class, did you find the topics redemable or did your vast experience blind you to the fact that others did not design algorithms for Bell Labs?

What’s really funny about your arguement is that by your standards, no one could be hired because they have to get experince from the wave of a magic wand, that is unless they are in good with the “Old Boy Network” (as per Marcus).

Honestly I think eleitism is one of the biggest problems with security.

One the one side: You suck because your are an ignorant blind person who knows nothing about security.

On the other side: You suck until you can own a Fortune 500 network with your sploits.

So when you try to learn something, the message is you suck because your not as good as me (for whatever reason).

Ya, that a really good message for security.

Anon-CISSP • July 21, 2006 10:47 PM

@Anonymous

You know, if you read through my previous statements I’ve said that certifications are not a good measure of experience, that CISOs should have a technical degree so they can effectively lead their group, and that the trade-schools turned degree mills are garbage.

Where is there elitism there?

Nowhere did I say people with certifications shouldn’t be hired. You have a habit of misrepresenting my statements into a position you can more easily attack. It’s called a “Straw Man Fallacy,” look it up.

As a final note, you seriously should stop worrying about what Marcus and others are doing and focus more on networking and your own professional development. How can you be a creative security person if you think anyone who publishes a book is a god to be revered? Skepticism and the ability to think critically are more important to a security person than any college degree or certification.

@nonymou5 • July 22, 2006 2:43 AM

I’ve said that certifications are not a good measure of experience, that CISOs should have a technical degree

You argued once about CISOs and that was not in the context of argueing with me. I never said I expect a cert alone will become a CISO. But if you need someone to harden Linux boxes, a SANS GCUX would be a good canidate to check out. At very least if they are Gold level you could read their paper (posted online) to see the process they used to do so. That could give you a better idea than lines on a resume.

Where is there elitism there
Examples:
They must have made their name on the security lecture circuit when I was busy working on algorithms at Bell Labs.
I agree that certs are basically worthless.
You know, if you read through my previous statements

Plus in general posters to security blogs generally like to insult those who are not up to thier standards. (Except Bruce himself, I like his gentle approach. Since Bruce understands that security is a human problem, it does not help to alienate the other humans)

I’ve said that certifications are not a good measure of experience
Yet you said…
Vendor certifications are great, if I managed a group that dealt with firewalls, I’d want everybody in that group to have the highest vendor certification for the firewall they use.
So you want someone to have the cert after getting the job.

I have been pointing out that with you no one would get the experience. Right now I work in making content for a policy compliance software. Since I am a user in the eyes of the company, I don’t have the opportunity to manage routers or the NIDS outside of buying hardware from e-bay and getting certs. So ya I have experience, just not in the role I would like to be at this time. Not all of us work for companies that are good with career development

You never answered my question about finding anything redeemable in a cert beyond “It’s definitely better than nothing when trying for an entry-level job.”

As a final note, you seriously should stop worrying about what Marcus and others are doing…

WOW!!!
So I can’t learn anything from the inventor of the proxy firewall and a pioneer of NIDS? I should not examine the lessons of those who have gone before me and gain from their insights from the books that they write? I should not learn the lessons of a network admin who spent years focus on the one topic of log analysis? I just don’t see how this is good advice.

and focus more on networking and your own professional development.

I also love the assumption that I have not been working on my professional development. Do know Jack Sh1|7 about me and already assume that I don’t stay up late at night and read books like the Stevens TCP/IP book or that I have a series of VMware images to stay current with differ OSes and tools. Also since I have been arguing for certs you can also imagine that I have been working on a number of certs.

How can you be a creative security person if you think anyone who publishes a book is a god to be revered

So I can’t have heros in my industry. Hey Bruce, I can’t admire you or learn anything from you as per Anon-CISSP.
Geez I guess no one ever lerned anything from W. Richard Stevens TCP/IP book or from the authors of “UNIX System Administration Handbook” or for that matter I guess I can’t learn a damn thing from this dude Bruce Schneier’s books.

Skepticism and the ability to think critically are more important to a security person than any college degree or certification.

Too bad skepticism doesn’t make it past the HR desk.

Skepticism is good. But at some point a decision has to be made. The best decisions are made when you listen to the best information you can find and evaluate it amongst all the choices. I believe it is wise to listen to the heros of the industry and learn from them. Isn’t that one of the reasons we all post on this blog?

Anon-CISSP, in all seriousness, I admire your success. I hope you retire a wealthy man. But for those of us who work for a company that has cooked the books, is under SEC investigation, sunk the stock and is now a big stain on the resume, we have to do whatever we can to make ourselves look better on our resumes. I glad you don’t have those kind of problems. But for me, I am doing what I can to improve my position in life.

Ben • July 22, 2006 8:13 AM

A the end of the film ‘Crossroads’ Joe Seneca’s character says to Ralph Macchio, a pro pos Ralph’s newly acquired appreciation of the blues, that you ‘gotta take it past where you found it’ .
So many people, in all walks of life, forget this and develop no more. So many people, in all walks of life, rote-learn only the same, simple blues riffs. Where ever you start, from a book, from ‘playing’, from having lessons, you have to take what you know further – use it, recombine it, add to it, take from it.
Certifications, in any feild, are shortcuts to the recognition to anothers personal authority and training/education in a given area. But, as we all know, a badge don’t make you a good cop. A porkpie hat and a mouth organ don’t make you play good blues.
If you make a mistake, learn and move on.
IMHO it is only by commiting to, practicing and following the ‘living truth’ of our feilds, that we can avoid the adherence to ‘dead dogma’ that it is all to easy for the unintiated to believe and the lazy to preach.
Or may’be I would say that being a qualified philosopher and certified network admin, who’s resume includes ‘failed musician’, firewall administrator and who’s job is persuading the unwilling that security audit’s and effective system monitoring are really quite important.
I am also studying for further vendor-specific and non-vendor specific security certifications (to back up my ‘on the job’ experience).

Anon-CISSP • July 22, 2006 10:34 AM

@@nonymou5

But if you need someone to harden Linux boxes, a SANS GCUX would be a good canidate to check out.

Yes, a good CANDIDATE, but the certification shouldn’t be the sole basis for making hiring decisions because it’s not a good indicator of experience. That’s the crux of Marcus and Bruce’s argument.

So you want someone to have the cert after getting the job.

Yes, technically competent people who get paid to support a product should receive all of the training possible for that product. With that training usually comes the vendor cert. This doesn’t mean that someone who is currently unqualified to work on firewalls should do an “exam cram,” pass the vendor test and become automatically qualified to manage the firewall. I know you have to see the difference here.

Not all of us work for companies that are good with career development

I mentioned earlier to get involved in Infraguard. Why not become active in ISSA, Infraguard, and other groups? For the cost of a SANS conference you can get probably 10 years membership in one of those associations. You could volunteer to do a presentation on whatever technology you’re trying to master. Someone who’s spoken about a technology in front of several hundred people carries a lot more credibility than a certification because you have to really know it to talk on your feet in front of an audience. You would meet dozens of people who could use your skill and might be offered a job doing something you’d rather do. Most of these associations would kill to have original material presented, lately the ones I belong to have been nothing but pseudo-sales presentations by vendors.

So I can’t learn anything from the inventor of the proxy firewall and a pioneer of NIDS?

Again, (see Straw Man) I didn’t say that. But you need to stop thinking that anyone who’s published a book is omniscient. Look at the errata for any of the books you read. People make mistakes, and the security person who can think critically is in a much better position than a security person who blindly trusts people. If Bruce came to you with a crypto algorithm that nobody has ever seen before and asked you to implement it for a sensitive process, would you trust it just because Bruce Schneier wrote it? What would Bruce do if someone gave him a new algorithm that hasn’t undergone peer review? If you think I’m saying that Bruce doesn’t know crypto, you’ve completely missed the point.

stay up late at night and read books like the Stevens TCP/IP book

That’s the best book on TCP, hands down.

Too bad skepticism doesn’t make it past the HR desk.

True, but people without critical thinking skills usually get winnowed out during the subsequent technical interview.

Anonymous • July 22, 2006 12:38 PM

@Anon-CISSP

Again, (see Straw Man) I didn’t say that. But you need to stop thinking that anyone who’s published a book is omniscient.

That’s funny also. I get accused of the Straw Man agruement, and yet you post one of your own (over and over). Did I ever call one of the people I mentioned “omniscient”, no. Did I ever call one of them a “god”, no. Only after I get accused of doing so did I decide to say that I have heros of the industry.

Even funnier are out of the original four, only one of them is known of his books, Chris Bretton. Marcus wrote one book and it did not sell well and I never read it. I respect him on his other achievements. Matry Rosche may have coauthored a book on Snort or maybe he just wrote the forward. Again, he is respected for Snort. Tina Byrd collborated with Marcus Ranum for over two years and help set some of idea considered standard for currently log analysis. She also worked for Counterpain at one point. Again I respect her for her achievements. I don’t believe she even has authored or wrote a forward on a book.

People make mistakes, and the security person who can think critically is in a much better position than a security person who blindly trusts people.

Another staw man by you. I never said I blindly trust them. My original arguement was there are people worth learning from at SANS. I gave a small list of these people. To that list your reply showed that you believe you could not learn anything from these folks. So were disagree. We are both in different positions in our lives. But just because I respect them does not in anyway say I have them as “gods”, hold them as “omniscient” or “blindly trust” them.

rue, but people without critical thinking skills…
Critical thnking begins with learning from others and evaluating were they agree and disagree with each toher. Then taking that knowledge and applying it to other situations and evaluating if it fits. Then at some point making a decision.

Yes, a good CANDIDATE, but the certification shouldn’t be the sole basis for making hiring decisions because it’s not a good indicator of experience. That’s the crux of Marcus and Bruce’s argument.

Finally I thnk we agree. I never said “sole basis” (Another Straw man by you) But I find in security forums in the last year I have had to defend my certs. Some folks seem to asscert that you are less qualified if you have a cert. As if making some effort to improve your career hurts your career. The ironic thing is (in the case of Marcus Ranum), is while he was puting certs down, he has also taught many times for SANS. So I am mearly pointing out that he must think there is something worth while at SANS.

pjm • July 23, 2006 9:29 PM

@Mike – mostly, I was concurring with Bruce’s assertion that, “I’ve met too many bad security professionals with certifications and know many excellent security professionals without certifications.”

Pauleto • July 24, 2006 9:09 AM

I’ve been involved with IT security for 5 years now, and like some of the posters here, I’ve noticed a trend in the people I’ve worked with. People who heavily favor certs and spend every waking moment obtaining them are great in the theory department, but it’s the people who just experiment and learn from trial and error in the real world that end up knowing what they are doing. I’ve seen this time and time again.
I’m being pushed to get the CISSP and I refuse because it will do nothing other than flower my resume. It’s a marketing tool for people to use — nothing more, nothing less. It does nothing to prove I know security.
Security is a process, not a product, and that product includes certifications. Learn by experience in the real world , not by reading a book.

Pat Cahalan • July 24, 2006 12:29 PM

@ Pauleto

I’m being pushed to get the CISSP and I refuse because it will do nothing other than
flower my resume. It’s a marketing tool for people to use – nothing more, nothing less.

So… you have already absorbed all of the information that is covered by the CISSP? You can learn nothing new from going through the process?

I brought this up much earlier in the thread, and I’m going to repost it now: people acquire technical know-how through four methods. They learn from academic programs (theory), certifications (procedures), real-world experience (theory and procedures interacting with real-world limitations), and mentoring (which provides a little of all three).

Some people in the IT world absolutely hate certifications. In my experience, most of these people are self-learners, often intelligent, and usually dismissive of structured learning. If it doesn’t come from experience, they figure, it’s worthless. I’ll reiterate another point I stated earlier -> experience can be a bad teacher, because there is no guarantee that the solution you implement in a real world situation is the best solution. In many cases, real world solutions suffer greatly from scalability issues, because they are a response to a real world scenario.

Another, byproduct of “certification hatred” is that those that dislike certifications also generally dismiss people who get certifications under the logic that they are people who are only interested in marketing themselves. Although this is certainly a non-trivial instance, this attitude is equivalent to dismissing people who don’t have certifications as being unskilled. It’s just plain foolish.

People interested in having a career (as opposed to just holding down a job) should be interested in all aspects of their workplace. This means having an academic grounding, learning through real-world experience, picking the brains of your coworkers, getting certifications to learn practices and/or theory they may not already possess, learning basic business rules like budgeting (this is probably one of the most glaring deficiencies in lots of IT professionals), polishing people skills (particularly communication skills), and even learning the terminology of other areas, like the dreaded business-speak.

You don’t need to become a pointy-haired boss, but if you want to be an effective IT person, you had better possess the skills necessary to take on other pointy-haired bosses on their own turf using their own language, or you’re not going to be able to pursue a security agenda.

@nonymou5 • July 24, 2006 3:18 PM

@Pat Cahalan

Thanks Pat for articulating some of the points I attempted to make. I know the CISSP gets a lot of abuse since the test is mainly memorization of terms, but as you pointed out can anyone here claim to have mastered all it’s domains?
On the same note, anyone can download Snort and run it. That person may even show some experience with running it in a production network. But it’s nice to have the baseline that SANS has for the GCIA. In this class the first 1/3 of the content is examining packets with TCPdump. Then using TCPdump as an ids. Only then when you have experinced the “manual way” to examine packets will you move on to snort. Then it covers topics such event correlation, troubleshooting false negatives/positives, classic attacks, and log analysis. While I am sure many people who are already sysadmins could have learned this on their own, others who want to get more involved in this area need a place to start from.

Another, byproduct of “certification hatred” is that those that dislike certifications also generally dismiss people who get certifications under the logic that they are people who are only interested in marketing themselves.

This is a bias I have faced. It does not help that many vendor certs and the CISSP have programs to “get a cert in a weekend”. This is still a reason I like SANS. Unless you are already a seasoned packet reader, I doubt you could just pass the GCIA without sitting down and running the excercises. On both tests (each SANS cert is two tests) on the GCIA, almost 1/2 the questions per test are a post of a dumped packet/s, a question about that packet/s and 5 answers to choose from. Sure it’s an open book test. But if you don’t already understand what you are doing, you will run out of time and fail the test. The open book part only really helps with the trivia like questions regarding the name of some classic attack. Also the those who have earned Gold status have also written a paper on a topic involved with that cert. The paper is posted on line so the interviewer could at least read the paper to get an idea of how the canidate can write about the topic. Is it perfect, no. Nothing is perfect. But it is certainly better than the “just passed a test” criticism.

Although this is certainly a non-trivial instance, this attitude is equivalent to dismissing people who don’t have certifications as being unskilled. It’s just plain foolish.

Exactly. Notice how I was praised for reading the Steven’s TCP/IP book, but criticized for my certs and listening to SANS speakers. Also when you consider the material for the GCIA was written by Stephen Northcutt (pioneer in NIDS), Judy Novak (Works for Sourcefire), Mike Poor (Used to Work for Sourcefire) and Marty Rosch (Founder of Sourcefire), why would that be worse than reading the Steven’s TCP/IP book? I see the Stevens book as giving me more depth on a specific topic, buit the GCIA material shows me the lessons learned from the pioneers of the industry for all aspects of intrusion detection. Where can I go wrong with that?

There is one part that’s tough to get…

mentoring (which provides a little of all three).

This is where a lot people in the security world who have no time for you. Hence the next best thing you can do is read security Blogs and the mail from specific mail lists to learn what they consider important.

m Dundas • July 25, 2006 10:03 AM

“@Bruce, is this a roundabout way to announce a new certification program of yours? :p”

If the above was true I’d really like to know. I would be more than willing to take a certification course(s) that were done by someone like Bruce.

I, like many on this blog have had ‘issues’ and mixed feelings with Certifications. I have a few certifications, routing,BGP, wirless, ‘sniffer’ etc. I have no security certifications, yet I’ve worked in the security field 10 years now (doing both research and implementation). Why am I not certified in security? The areas I am certified in, I actually took because I researched them to know that I would be better for it, both in understanding and in practical. They were ‘recommended’ by people I respect in the respective fields. Up until now, many of the security courses had none of those qualifications. For example, one ‘popular’ security course that many employers request, two individuals I used to work with (they were not technical at all) received in a weekend. They wouldn’t know what a packet was if it hit them on the head … yet apparently they are now more qualified than I was to audit security. After that, it became a red flag for me. Any employer ‘looking’ for a course as a filter was immediately suspect in my books. I expected I would not be happy working for them, because I know how I work and if that is their ‘basis’ then there is already a fundamental problem.
As stated, things are changing in the certifcation world and I am again looking into these certifications. However, I’d rather it be endorsed by someone beyond a vendor, employers, or the general ‘security public’ as ‘good’. If I’m going to spend the money and the time to get certified then endorsed by Bruce or someone with equivalent repsect in the industry would be awsome.
With Bruce I bet would be an excellent certification, just because of how he thinks / assess etc. in my opinion.
–mike

Bruce Schneier • July 25, 2006 11:12 AM

“@Bruce, is this a roundabout way to announce a new certification program of yours? :p”

No. I have no plans to certify anyone for anything, or anything for anyone.

Although maybe I should think of it — there’s big business in giving people random letters after their name.

Rich • July 25, 2006 1:57 PM

@@nonymou5

I know where you’re coming from and it’s very difficult to break into the security field with little experience. Companies don’t give a hoot about security unless there’s regulatory or financial impact — my first job (UNIX Programmer/Administrator) was at a company which was a victim of Mitnick, and the only thing they did was authorize me to start running SATAN scans.

Here are some of the things I’ve learned during my 10+ years in InfoSec.

  • CISSP – learned some things prepping for the exam. Passed the test in 90 minutes and walked out confident I did well. Most employers don’t really care if you have it or not in my opinion. What counts more is experience, and since you need a few years experience (+ a college degree) before taking the CISSP, it’s almost self-defeating. Yes, some of the worst InfoSec people I’ve known were CISSPs. In my experience, the worst ones always seem to tag “CISSP??? on every document and business card they touch in an attempt to establish credibility, maybe due to this the bad ones are more visible?
  • College – I quit college to start my first job and went back to school a few years later and finished a BA in Computer Science/Mathematics from my state university. Best thing I could have done.
  • SANS – I’m actually very good friends with several former SANS speakers, many of which are authors of books. I think that SANS is good at teaching a lot of security as it applies to system and network administration. If you want to be a security-oriented system/network guy, then SANS is probably a good start.

If you’re not working in IT now, you might want to make a jump to someplace where you can get hands on with systems/networks before trying for a full-time security spot. The field may be too specialized to just jump into full time. It’s hard to get a security position with a strong IT background – if you have no IT background it may be even tougher. Think about where you want to be 10 years from now and start planning on getting there.

Rich • July 26, 2006 12:58 PM

@Pauleto

Yes, I agree on your points about the CISSP. I passed the exam over 7 years ago — before there were boot camps and books dedicated to the exam. Back then you were given a subject list of topics and you had to make sure you knew the material on your own. Now, you can buy one book that will allow you to pass the exam by memorization.

Personally, I don’t brag that I have the CISSP (not on my business cards, in my email signature, no coffee mugs, etc). I don’ think anyone in the security community would hold it against you if you had the CISSP, but narrow-minded managers may hold it against you if you don’t. While I’d be happy to make a case with you that those narrow-minded PHBs are someone who I’d never want to work for, sometimes you get a new CIO who will make a promotion decision on whether or not you have the CISSP on the basis that trade mags like “Information Security??? say that a CISO needs a CISSP.

It’s not fair, but it’s the way office politics work. If two people are competing for the same spot, a poor manager (which in my management experience is like 75% of them) will promote the person with more credentials because if the promotion turns out to be a train wreck they can always say that the person was certified and that they made a sound management decision based on the information they had.

Marcus J. Ranum • September 13, 2006 12:59 AM

@anonymous:

Also since you have taugh SANS classses for both Securing LAMP
and for Log Analysis I would think you would have endorsed SANS
certifications.

Teaching and learning are valuable activities in and of themselves!!!! If a job applicant comes to me and tells me “I took classes from Dan Geer on system design and studied physics with Richard Feynman” that conveys specific information that I can ask them about and assess them based on. If they just say “I have a SANS (or whatever) certification” the only way I can evaluate their knowledge would be to dig into what that certification meant.

My point (and Bruce’s, really) stand: a certification or a degree is a convenient way of heaping a bunch of details into a larger “profile” on which you can base a judgement. In my view, basing a judgement on something so coarse-grained is sheer laziness! If you are assessing mathematicians and all you do is look whether they have a PHd you’re lazy. You should be asking what they did their dissertation on, what their interests are, etc. Hiring based on a certification is outsourcing the hiring process to the certifier. That’s just stupid!

mjr.

Pat Cahalan • September 13, 2006 1:57 PM

@ Marcus

I don’t think anybody disagrees with your last post. I certainly don’t 🙂

But as I pointed out above, that doesn’t mean that the process of getting a certification is useless, or the presence of a certification on somebody’s resume indicates a problem.

There context of your point/counterpoint seemed to indicate “certifications are entirely useless”, as opposed to “certifications alone aren’t necessarily a good indicator of employability”.

I agree totally that basing a judgement on an alphabet soup is foolhardiness. On the other (employee) hand, persuing certifications can be a good thing in and of itself.

Anonymous IT Pro • April 23, 2008 6:29 PM

People that have certifications, education, experience, understand business objectives, and have a positive attitude make more money and typically love what they do.

I am one of these people, and I know others like me. There is no “magic bullet” to a career.

Harvey MacDonald • July 5, 2012 9:02 AM

It’s pretty simple. The certification exists so that the jobseeker can legitimately put color on his/her resume, which causes the recruiter to spend more than 6 seconds staring at yet another black/white page detailing the qualifications of a complete stranger. The interviewer has the responsibility of testing the jobseeker in practical ways, such as this: “Hey you. Build a LAMP system. Patch it, harden it, and then write up an assessment of the real risk of using it in .”

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.