NSA Implements Two-Man Control for Sysadmins

In an effort to lock the barn door after the horse has escaped, the NSA is implementing two-man control for sysadmins:

NSA chief Keith Alexander said his agency had implemented a "two-man rule," under which any system administrator like Snowden could only access or move key information with another administrator present. With some 15,000 sites to fix, Alexander said, it would take time to spread across the whole agency.


Alexander said that server rooms where such data is stored are now locked and require a two-man team to access them -- safeguards that he said would be implemented at the Pentagon and intelligence agencies after a pilot at the NSA.

This kind of thing has happened before. After USN Chief Warrant Officer John Walker sold encryption keys to the Soviets, the Navy implemented two-man control for key material.

It's an effective, if expensive, security measure -- and an easy one for the NSA to implement while it figures out what it really has to do to secure information from IT insiders.

Posted on July 24, 2013 at 6:18 AM • 41 Comments


HotJamJuly 24, 2013 6:26 AM

So, no more leaks from SysAdmins...

But not everyone in the NSA is a SysAdmin...

More whistleblowers please!

AspieJuly 24, 2013 6:44 AM

Is this proof against a cooperative effort in future? I imagine recruiting a compliant accomplice from a pool of ... uh ... one, will be near-impossible but I doubt they can put 1.5m people under close cross-watch.

I don't think Mr. Snowden will the the last "stranger in the house" the NSA will have to worry about.

SamourJuly 24, 2013 6:46 AM

Maybe initially, but long term and eventually when the two people establish a close relationship and one of them decides to become a whistleblower, the other one may help him out. Unless they frequently change the teams.

everyday.guyJuly 24, 2013 7:12 AM

Any organization doing highly immoral things will always have the insider threat.

And they will always lose in that fight.

Richard S. WestmorelandJuly 24, 2013 7:16 AM

Until they dedicate a team of analysts to watch the accesses, they haven't solved the problem.

JurgenJuly 24, 2013 7:26 AM

What was it again with the Cold War issue of two keys for nuke launch..? If the other guy wouldn't want to fire, shoot him and turn both keys yourself. I.e., let the compliant guy log in (maybe even watch his/her (sic) keystrokes; separatre rooms would be too cumbersome and not protecting against intruding bystanders in that other room), then shoot him and have free access.

Also, the night shift during Christams will be somewhat difficult to schedule ..?

bJuly 24, 2013 7:33 AM

Good sysadmins do not work in two men team. It is simply not their culture. Accordingly their productivity will significantly decrease. A two men sysadmin team will be less productive than one good sysadmin alone.

The good aspect of this is that it throws a lot of sand into the machine. Now the NSA will have to double its numbers of sysadmins. Find them in the current rather anti-NSA climate, get them through security clearance and train them on those systems will take years.

The NSA will have to slow down or stop lots of new stuff until they get this implemented.

In the end the complex will be no more secure than it was before. People will work around the new procedures and find quicker ways to get their work done. You have to trust somebody to get the whole machine to work.

maxCohenJuly 24, 2013 7:50 AM

"Alexander said that server rooms where such data is stored are now locked"

Hooray, they finally locked the door!

officerXJuly 24, 2013 8:05 AM

No women @ NSA then?

But seriously, given the right incentives I am sure whistle-blowers could easily work in teams of two, or more.

WinterJuly 24, 2013 8:19 AM

All these new rules are just more prove that everything the NSA and others said about privacy being protected were lies.

There simply was nothing that protected civilians from any and every abuse by NSA employees and contractors.

I wonder how many contractors had a side business spying for "external" customers? Say, the mob, or politicians spying rivals?

WhatDidYouExpect?July 24, 2013 8:31 AM

So, it boils down to who is watching the watchers? An example of this is


where the NSA says it can't look at its own employees' email. And why is that? If they can look at everyone else's email, why not their own? Well, that is probably due to NOT collecting ANY data on their own email. Which, begs the question...what other communications are they NOT collecting data on? CIA, FBI, Congress, the Administration, political contributors, the security-industrial complex, etc? Most certainly, they are not seeing any data about the telcom providers that are doing their bidding.

In all likelihood, it will come to pass that so much data is being collected, that the technology in use will become antiquated and need to be replaced. However, while new technology will be smaller, faster, and cheaper, there won't be time enough in the day (or weeks or months or years) to convert old data to any new technology, and as such, the data will effectively rot in place, with perhaps more than 99% of it never being used.

Eventually, the cost effectiveness of this wasted effort will catch up to the program, and like every other government program that lacks value and accountability, they will stop spending money on it and future generations will wonder about the stupid people that started this. The possibility also exists that the people spawing the data, will stop doing so, mostly because they will eventually lack the wherewithall to do so (no jobs, no money, sick, dying, or have no reason for using any facilities that they now collect the data from).

NobodySpecialJuly 24, 2013 9:37 AM

So finally, our security agencies have learned from East German border guards.
If one guard defects - you shoot the rest of the watch.

Nick PJuly 24, 2013 9:40 AM

So it's basically Pair Programming for Sysadmins? ;)

(Subtitle: nuclear analogy is a very POOR metaphor)

There's not enough in this article to know if the system will be effective. It's easy enough to slip some innocent looking, yet subverting, commands in when your partner isn't looking. I mean, they'd need to be doing the equivalent of pair programming with equal knowledge of security-relevant configurations/commands and absolutely *zero* alone time. Not even enough for one loaded command that could be used as an anchor point for future intrusion.

The other risk is collusion. People who work so closely that they're practically entering and watching the same commands for hours at a time develop a connection. They might start trusting each other, deferring to each other, etc. The work relationship necessary to be productive, along with any personal one they develop, might undermine the effectiveness of two person security.

So, the nuclear metaphor is entirely inappropriate. Rather than a quick moment together, these people would have to spend a whole lot of time together in one place doing their work. This weakens the protection of two person security.

One option might be to rotate staff every few months for critical systems to discourage the people from getting too close to one another. Another is to set up physical monitoring and logging connected to remote admins. This makes them easier to swap out. Plus, all their conversations happen over the network increasing risk of collusion.

@ Jurgen

"What was it again with the Cold War issue of two keys for nuke launch..? If the other guy wouldn't want to fire, shoot him and turn both keys yourself. "

There was more to that scheme. The locations for the keys were supposed to be separated from each other too far away for one person to use both simultaneously. That was a neat idea. In practice, I think they skimped on that one at many, most or all sites.

DMSJuly 24, 2013 10:08 AM

Years ago I was complaining to a lawyer about the inefficiency of government.
His reaction was "Thank god!"
The only thing which hinders tyranny is that the government is too screwed-up to be truly effective.

He was part joking, and so am I as it's perfectly possible to have a tyrannical government (and yes it can be described in many ways) and still be inefficient such as jailing a suspect who has a similar name etc etc.

But there is some grim humor that NSA claims (yes, it's a mere claim as it is too absurd to believe that it is true) that NSA doesn't have any way to filter email from ALL NSA employees.

DannyJuly 24, 2013 10:17 AM

"It's an effective, if expensive, security measure..."
I disagree with you Bruce. It's a bottleneck, and like history already taught us already at Pearl Harbor, when they employed the same "two man" rule on weapon stashes because of feared of sabotage - it led to disaster. In the morning of Japanese attack soldiers were unable to fight back because they could not get their hand on weapons. Hence the Pearl harbor was totaled.
Now imagine this on cyberwar, and with the speed the information is traveling versus the speed of Japanese aircraft compared to human reaction this will be exponentially worse then Pearl Harbor.

Clive RobinsonJuly 24, 2013 10:18 AM

It won't work with the current systems.

The reason two-man works for things like KeyMat (crypto keys) is that in a military and other Government environment in essence you are moving a physical object in a physical security environment and simple human observation to a well thought out proceadure is all that is required.

Being a systems administrator often involves "fixing things" for which there is no let alone well thought out proceadure. Further large data sets are frequently not moved as "physical objects" and this has all sorts of implications due to near "zero cost to copy" in what is in effect a non physical space where direct human observation is not possible.

To put it in a mundane way which most readers will be aware of the *nix and Windows OS's have "pipelines" for Interprocess communication (IPC). In *nix these are generaly system memory only IPC's, however in many MS OS's and attendant applications the pipeline is done as a file store IPC in that in effect it writes it to disk then reads it back in to the next process (this is a legacy issue to do with the non-multitasking design of DOS that carried forwards into Windows and NT).

Now ask yourself what is going on with regards to this IPC, ie what has access to it and at what level. In a system memory only system any kernel level process has access to the shared memory. In addition a file store IPC the Kernel and various low level drivers have access to the file store. Thus there could be many processes that are effectivly hidden that could get at the data.

But it gets worse it's unlikely that any usage of the file store will be "cleaned up" securely this you could easily have partial or full remenants of the data available at some later point.

Now these are well known issues and have been known since the 1960's --if not earlier-- and a number of specialised secure OS's were designed to deal with these issues.

However in the mid 1980's and certainly quite openly in the 1990's these OS and the hardware they ran on were ditched/replaced with low cost Commercial Of The Shelf (COTS) systems for what were financial not security reasons. That is the inbuilt security was ditched, and replaced with internaly very insecure systems. This was reasoned to be OK because security could be moved to the perimiter of the computer or network it was connected to.

The problem is that whilst the normal system users are in effect "outside" of the system perimiter the system admin is not.

The cost of resolving this blundering "COTS policy" is going to be high and it will be interesting to see how they do it and what impact it will have on all those "Microsoft trained" users and sysadmins.

One things for sure it ain't going to be elegant, and likewise it ain't going to be painless or quick...

Petréa MitchellJuly 24, 2013 11:01 AM

When I visualize two sysadmins having to go into the server room together I keep seeing one of these things happening:

1) Their supervisor expects that since there are now twice as many of them, the work should take half as long, so they will be pressured to split up and not actually watch each other.

2) If there really is strong adherence to putting two people on a one-person job and requiring the second one to stay with the first, it's going to be one doing the work and one standing there playing with their phone rather than closely monitoring them.

Bob TJuly 24, 2013 11:48 AM

Maybe they should just stop doing underhanded and illegal cr4p that people feel the need to blow the whistle on.

Greg PJuly 24, 2013 1:15 PM

"Alexander said that server rooms where such data is stored are now locked..." I certainly hope that isn't a new part of their procedures.

@Bob T: Your "you don't need to worry if you have nothing to hide" assertion sounds just as stupid as it does when pointed in the opposite direction.

OJuly 24, 2013 1:22 PM

Security will increase until someone insists on actually getting some sort of useful work done.

OJuly 24, 2013 1:30 PM

Though I suppose that could be a while, what with this being the government and all.

canuck-by-choiceJuly 24, 2013 1:55 PM

Nick P add-on

My ex worked for Revenue Canada's Audit Branch for 6 months. None of the auditors worked for them longer than 6 months at a stretch; then they went back to their usual positions. Too tempting to blackmail someone eligible to become your victim.

Duncan KinderJuly 24, 2013 2:41 PM

If I were a Mexican drug lord, I could think of methods of persuading two - or even ten - men to do my bidding.

JasonJuly 24, 2013 3:18 PM

Did the NSA chief just leak the number of sites that need fixing (15,000)? Is that a new disclosure or was it already leaked by Snowden?

NargunomicsJuly 25, 2013 12:38 AM

Now that is an interesting problem if one is using SELinux or a similar MAC-based permissions system.

To install software or update it on an SELinux system involves not just the Systems administrator, but also the software updating/installer and as many other subsidiary subadmins as considered necessary - though the sensible consider that only the sysadmin and the update subadmin are necessary.

In theory it could work quite well. One sets up the sysadmin role to permit the update subadmin role to function; the update subadmin role does his thing, then signs off; the sysadmin quits his role and the system rolls on.

In theory, that is. I suspect the NSA, as a result of its vastly increased "focus" - rather like someone "focusing individually" on the Virgo Supergroup, or "focusing personally" on the combined populations of India and China - has increased its sysadmin subdivisions, and now each of two sysadmins will have to "focus" on doing two or three individual things at one and the same time. As Robert Anton Wilson observed at least once in The Illuminatus Trilogy, "Never whistle while you're pissing."

And as has been observed, it does nothing to counter the effect that several decades of education, training, brainwashing, whateveryouchoosetocallit, will have on the sysadmins. Snowden did what he did because he was taught to regard the world of humanity in a certain way, and the way the NSA and the US Federal Govt was behaving contradicted what he'd been taught.

OnTheWaterfrontJuly 25, 2013 5:50 AM

This is a boon for defense contractors. They all got their contract amounts doubled.

kashmarekJuly 26, 2013 12:55 PM

They don't want to protect the data (or metadata). They want to protect the watchers, the deciders, the (inane) policy, how they use the data, or if it any of the data is useful. If the data (or metadata) is protected from anything, it is protected from the accused ever using it to defend themselves or prove their innocence. Further, such data (or metadata) will be held to be secret so it can't be seen by judge or jury, just in case it might embarrass those that collected the data or fail to prove guilt.

SchneieronSecurityFanJuly 26, 2013 9:41 PM

Look for this feature to be implemented in operating system software soon. Certain functions will require authentication from more than one user account; either more than one administrators or a pre-selected group of authenticators specifically for the function in question.
It would be similar to a business check with two signature lines.

The NSA will probably have to hirer additional contractors in order to implement this new policy.

Concerning the nuclear missile launching scenarios: Didn't the keys need to be turned by TWO pairs of launchers (both pairs being in different locations to prevent one pair of launchers from setting off missiles on their own)?

In keeping with the movie "Wargames", after the next security breach the NSA will then move to centralize all access accounts just like the centralized WOPR computer from the movie.

Joshua GruberJuly 28, 2013 10:33 AM

One of the points Snowden made was that, contrary to government assurances, many people could view citizens' private conversations without legal, or even procedural, permission. This, accidentally and merely as a side-effect, actually takes steps to address that issue.

djJuly 28, 2013 11:52 PM

What happens if one of the sysadmins had a bad stomach and needs to rush to toilet in the middle of a key transaction ? Would the other admin be unable to work properly alone ? It is the same phenomena of using multiple encryption keys to encrypt some data. Would it be really that useful ? I guess it's an issue of trust and inevitability.

hahaJuly 30, 2013 3:18 PM

Why would any sysadmin accept this new regime instead of simply quitting? Now they're going to have to work with a partner, watch that person _continuously_, and if that person still manages to leak something their partner will also share in the blame. It sounds like the work environment just became significantly less productive, more stressful and more risky for those sysadmins. If that was my job I would think about quitting and going off to the private sector for a job that was less hassle.

They'll need twice as many sysadmins, but the overhead of each one will be higher--perhaps a lot higher. The total cost to administer the systems might jump to 3x-4x of what it was before the policy. (Which would be great, of course, except that its our American dollars the NSA is pissing away on these programs.)

rufo guerreschiAugust 26, 2013 4:56 AM

The same 2-man rule concept could be used to guarantee services for ordinary citizens that provide ultra-high security and privacy, while complying with the Constitution, as assessed by ad hoc citizen juries made of randomly selected users.

For instance, any physical access to the server room (CivicRoom), where all service servers are located and all new devices are flashed, is physically conditional to the presence of at least 5 citizens or users randomly-selected and/or with conflicting interests.

The same idea could be implemented by NSA, mandated by proper legislation, so that all initial code and hardware running NSA services is publicly known, and after that all access to NSA server rooms would need to have a 5-citizens jury in addition to 2 admins.

Wanna help?


Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of IBM Resilient.