Security Information Management Systems (SIMS)

The computer security industry is guilty of overhyping and underdelivering. Again and again, it tells customers that they must buy a certain product to be secure. Again and again, they buy the products -- and are still insecure.

Firewalls didn’t keep out network attackers -- in fact, the notion of "perimeter" is severely flawed. Intrusion detection systems (IDSs) didn't keep networks safe, and worms and viruses do considerably damage despite the prevalence of antivirus products. It's in this context that I want to evaluate Security Information Management Systems, or SIMS, which promise to solve a serious network problem: log analysis.

Computer logs are a goldmine of security information, containing not just IDS alerts, but messages from firewalls, servers, applications, and other network devices. Your network produces megabytes of these logs every day, and hidden in them are attack footprints. The trick is finding and reacting to them fast enough.

Analyzing log messages can determine how the attacker broke in, what he accessed, whether any backdoors were added, and so on. The idea behind log analysis is that if you can read the log messages in real time, you can figure out what the attacker is doing. And if you can respond fast enough, you can kick him out before he does damage. It's security detection and response. Log analysis works, whether or not you use SIMS.

Even better, it works against a wide variety of risks. Unlike point solutions, security monitoring is general. Log analysis can detect attackers regardless of their tactics.

But SIMS don't live up to the hype, because they're missing the essential ingredient that so many other computer security products lack: human intelligence. Firewalls often fail because they're configured and maintained improperly. IDSs are often useless because there's no one to respond to their alerts -- or to separate the real attacks from the false alarms. SIMS have the same problem: unless there's a human expert monitoring them, they're not defending anything. The tools are only as effective as the people using them.

SIMS require vigilance: attacks can happen at any time of the day and any day of the year. Consequently, staffing requires five fulltime employees; more, if you include supervisors and backup personnel with more specialized skills. Even if an organization could find the budget for all of these people, it would be very difficult to hire them in today's job market. And attacks against a single organization don't happen often enough to keep a team of this caliber engaged and interested.

Back in 1999, I founded Counterpane Internet Security; we sell an outsourced service called Managed Security Monitory, in which trained security analysts monitor IDS alerts and log messages. Because of the information our analysts received from the network -- in real time -- as well as their training and expertise, the analysts could detect attacks in progress and provide customers with a level of security they were incapable of achieving otherwise.

When building the Counterpane monitoring service in 1999, we examined log-monitoring appliances from companies like Intellitactics and e-Security. Back then, they weren't anywhere near good enough for us to use, so we developed our own proprietary system. Today, because of the caliber of the human analysts who use the Counterpane system, it's much better than any commercial SIMS. We were able to design it with our expert detection-and-response analysts in mind, and not the general sysadmin market.

The key to network security is people, not products. Piling more security products, such as SIMS, only our network won't help. This is why I believe that network security will eventually be outsourced. There's no other cost-effective way to reliably get the experts you need, and therefore no other cost-effective way to reliably get security.

This originally appeared in the September/October 2004 issue of IEEE Security and Privacy Magazine.

Posted on October 20, 2004 at 6:03 PM • 18 Comments

Comments

Michael ChermsideOctober 21, 2004 9:33 PM

Minor editing nit in final paragraph: "Piling more security products, such as SIMS, only our network won't help." should probably be "Piling more security products, such as SIMS, on our network won't help." ("only" -> "on").

John BlackOctober 22, 2004 12:18 PM

This argument bothers me: No product contains human intelligence, so no product can do what a human could do.
My objection is that it is human intelligence that creates software. In many cases, software does what it does better than any human can - because of the great human intelligence creates it. I heard recently that most commercial airlines are now landed by computer software. I trust my spell checker rather than my own spelling ability. The list of cases where I trust software more than humans is large.
So why are humans reading vast amounts of boring log data going to be better, faster at seeing patterns of violation than the best software?

Sean GallagherOctober 22, 2004 6:07 PM

Bayesian spam filters get better at identifying spam over time, as they accumulate more human guidance.

Without invoking that dreaded word, "A.I.", would it be feasible to build a SIMS system that got better at identifying real security breaches based on the actions taken by its human operators?

That way, the system could coast on its accumulated experience for a few hours each day, say during off hours, and make it easier to staff the security group.

MarcOctober 22, 2004 7:05 PM

While I agree that the human factor cannot, at least at this time, be replaced, I think that SIM provides other value beyond the enhanced detection possibilities (capabilities?)

With SIM, closing the time gap between incident occurrence and resolution becomes better. Too often, security incidents have too many unique elements – where, what, the data involved, etc. By utilizing SIM as part of an overall response solution, it should be easier to contain or reduce impact.

For our companies use of SIM, we anticipate the biggest bang-for-the-buck will come from being able to repeat investigation processes (i.e. what did we do last time? Grep here, spreadsheet there, log data everywhere) and provide a more concise interface into the volumes of all that security data.

Oh, and by the way, we might just see some things we couldn’t before…

dagoOctober 25, 2004 10:18 AM

First, one question : if, as you said, SIMS aren't enough and needs human intelligence, counterpane could open up and give your SIM software for free, as the specialists analyzing the data is the real asset.

Second, a logical continuation of this (human vs software) would seems to balance in favour of in house security, as only insiders have real knowledge and contact with the humans which are using the IT infrastructure. Outsourced SIM services are just, for the company buying it, a very intelligent SIM ... not anything more than a software...

omeronOctober 25, 2004 12:55 PM

I think the problem lies elswhere. SIM is a detection tool. It is suppossed to be a second layer, above the prevention layer. Usually, the prevention layer isn't good enough and so you get a lot of garbage. Organizations are trying to compensate for the lack of prevention tools and methodolgy by implementing SIM. That's the real problem - it's being lazy and sloppy.

Stuart WrightNovember 11, 2004 6:21 AM

General Comment - completely agree with your sentiments on overhype of products - frankly I blame the 'big-4' who are too lazy to offer real advice when auditing customers - the question is how to get best practice security to reach compliance - the answer was - buy and implement IDS - or these days IPS!! Of course it doesn't work, they didn't ask or answer the right question. Now the answer is SIM or SEM or whatever you want to call it. Any product/solution bought for the wrong reasons is doomed to failure - because it won't get the priority, care and attention that anything needs for it to work. You don't get a nice garden because you own a spade!

The question should be how can I reduce my risks but still stay in business - not gain compliance - compliance is one big load of nothing. It just means you do what the guy down the street has been told to do. It means drive at 30mph on the motorway (OK freeway) because one guy once went 40mph and crashed.

Wow, amazing how a quick post can turn into a rant!


stefan burschkaNovember 17, 2004 1:17 PM

Bruce,

we met at the RAID in Nice. I listened
to your talk there, and I couldnt agree more with
you. Again thanx for this article.

We are the safeguard crew here and we're fighting
our way against the hype as well.

We tried a lot with AI, maybe you saw our poster there..... The bottom line is: the human admin
has to be supported, sometimes he's the best
anomaly detection and the best correlator himself.
And deficiencies in the human admin chain can't
be compensated by technology. You can have the
best SIM, or whatever hype word will be invented
in future, when the delays in the reaction chain
are in the range of hours to weeks it is just plainly
useless

cheers

stefan

Matthew McCartyDecember 27, 2004 4:10 PM

Has it occurred to anybody here that he sells an outsourced security service and he is touting as the only solution, an outsourced security service?

He also comes very well near saying that only HIS highly trained, specialized, super-intelligent analysts can do the job.

Translation -- "Hire my firm."

This guy is a security salesman not an expert.

ChrisSeptember 6, 2006 1:56 AM

If I may ask, what are the top 10 SIMs? Or who are the vendors who manufacture this SIMs? So that I can choose.

Thanks,

chris

Yogesh BadweApril 27, 2007 9:38 AM

The Top vendors in the SIMS market are :-

Novels e-sentinel
Netforensics
Trigeo
IBM
ArcSight

MeatpieandtattersNovember 7, 2007 4:51 PM

Top vendors as per Gartner? However they are ranked, considering that the least expensive product starts at $20,000 dollars makes them un-affordable for most SMB businesses.

ITD DanNovember 23, 2007 6:22 AM

SIMS can play a role in a defence in depth solution, but like most configurable items, they need to have their business logic provided by those analysts operating them.

As with so many secuirty solutions, configuration is often overlooked.

If tuned and used correctly, they do provide a tool that makes targeting specialist, resource hungry data forensics capabilities possible.

But they are expensive, and only really viable on high impact secuirty solutions.

New itFebruary 6, 2009 6:48 PM

i have issues trying to get Netforensics
(sim one) to run on the network i work on, where can i get a reference book or sim for dummies type of book, any help would be nice... thaks

Secret PatrolJune 25, 2009 3:19 PM

This is the most important part of this article - "The key to network security is people, not products. Piling more security products, such as SIMS, only our network won't help. This is why I believe that network security will eventually be outsourced. There's no other cost-effective way to reliably get the experts you need, and therefore no other cost-effective way to reliably get security."

Kevin GetsMarch 19, 2010 12:00 PM

Bruce has always "up played" his developments and "down played" his competition in his blogs and articles. He finds nothing wrong with his products. Why aren't Bruce products the monopoly? Because his has flaws too, but he won't advertise that.

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc..