NSA Collects MS Windows Error Information

Back in 2013, Der Spiegel reported that the NSA intercepts and collects Windows bug reports:

One example of the sheer creativity with which the TAO spies approach their work can be seen in a hacking method they use that exploits the error-proneness of Microsoft's Windows. Every user of the operating system is familiar with the annoying window that occasionally pops up on screen when an internal problem is detected, an automatic message that prompts the user to report the bug to the manufacturer and to restart the program. These crash reports offer TAO specialists a welcome opportunity to spy on computers.

When TAO selects a computer somewhere in the world as a target and enters its unique identifiers (an IP address, for example) into the corresponding database, intelligence agents are then automatically notified any time the operating system of that computer crashes and its user receives the prompt to report the problem to Microsoft. An internal presentation suggests it is NSA's powerful XKeyscore spying tool that is used to fish these crash reports out of the massive sea of Internet traffic.

The automated crash reports are a "neat way" to gain "passive access" to a machine, the presentation continues. Passive access means that, initially, only data the computer sends out into the Internet is captured and saved, but the computer itself is not yet manipulated. Still, even this passive access to error messages provides valuable insights into problems with a targeted person's computer and, thus, information on security holes that might be exploitable for planting malware or spyware on the unwitting victim's computer.

Although the method appears to have little importance in practical terms, the NSA's agents still seem to enjoy it because it allows them to have a bit of a laugh at the expense of the Seattle-based software giant. In one internal graphic, they replaced the text of Microsoft's original error message with one of their own reading, "This information may be intercepted by a foreign sigint system to gather detailed information and better exploit your machine." ("Sigint" stands for "signals intelligence.")

The article talks about the (limited) value of this information with regard to specific target computers, but I have another question: how valuable would this database be for finding new zero-day Windows vulnerabilities to exploit? Microsoft won't have the incentive to examine and fix problems until they happen broadly among its user base. The NSA has a completely different incentive structure.

I don't remember this being discussed back in 2013.

EDITED TO ADD (8/6): Slashdot thread.

EDITED TO ADD (8/14): Adam S, a former Microsoft employee, writes in a comment that this information is very helpful in finding zero-days, and cites this as an example. He also says that this information is now TLS encrypted, and has been since Windows 8 or 10.

Posted on August 1, 2017 at 6:00 AM • 89 Comments

Comments

Garrett BAugust 1, 2017 6:50 AM

My question is... How are they facilitating that interception, assuming the endpoint is HTTPS? Think the NSA issued themselves a *.microsoft.com cert? Sounds like Microsoft should be using cert pinning.. They have their own CA, it'd be easy enough to only trust their root cert if the machine isn't configured to send it's error reports to an alternative location. Also shows the growing importance of DNS security..

Jeff WilsonAugust 1, 2017 6:56 AM

This is one of the Snowden nuggets that really hit home for me when I read about it back then. As an IT Pro, I've seen my fair share of these mothership (as in, report back to the) error reports dating back to my earliest days in IT.

For whatever it's worth, I don't see them anymore with Windows 10/Server 2016. Microsoft has seemingly taken that ad-hoc approach to error reporting out in favor of comprehensive (and controversial) telemetry that is flowed back to the mothership over https.

On the plus side, as an IT pro, I can use many of the same tools Microsoft does to ingest, parse and analyze such telemetry...The Office Suite, for example, has a wide-ranging set of telemetry settings that allow me to capture Office performance and problems (as well as document/spreadsheet problems) across an entire organization of domain-joined PCs.

Safe to say the NSA has moved on from relying on these sort of ad-hoc just in time debugging messages.

Paul CoddingtonAugust 1, 2017 8:00 AM

Some of the data sent out will be a memory dump that may contain sensitive information, such as the content of a file that was being worked on at the time.

An interesting question that I do not know the answer to would be whether or not user credentials could be harvested from a lucky dump?

Or does Microsoft encrypt the memory dump part and NSA only sees the metadata?

Not a reliable way to gather information, but still potentially disturbing news for users.

vas pupAugust 1, 2017 8:32 AM

@all

Deception tech helps to thwart hackers' attacks:
http://www.bbc.com/news/technology-40751656
"We create a shadow network that is mimicking the real network and is constantly changing," he said. The use of so-called deception technology has grown out of a realization that no organization can mount perfect digital defences. At some point, the attackers are going to worm their way in. Given that, said Mr Bach, it was worth preparing for their arrival by setting up targets that are simply too juicy for the malicious hackers to ignore once they land and start looking around.

All article is informative.
Clive, could AI be utilized to analyze and separate fake network/servers/honey pots and real network?

parabarbarianAugust 1, 2017 9:43 AM

I do not assume the NSA is doing this on their own. Back in 2013 it was revealed that Microsoft collaborated with the NSA to provide access to encrypted data on Outlook.com, SkyDrive and Skype. If Microsoft is willing to do all that, a little bit of innocuous seeming metadata is pretty minor.

Some GuyAugust 1, 2017 9:48 AM

No doubt the NSA may also suppress error reports from flaws it intends to exploit en-route to MS.

Me Too?August 1, 2017 9:56 AM

Does anybody have knowledge and/or references of Apple bug reports being collected by third parties in the USA or elsewhere?

Does anybody have knowledge and/or references of Linux bug reports being collected by third parties in the USA or elsewhere?

Does anybody have knowledge and/or references of Unix bug reports being collected by third parties in the USA or elsewhere?

Does anybody have knowledge and/or references of iOS bug reports being collected by third parties in the USA or elsewhere?

Does anybody have knowledge and/or references of Android bug reports being collected by third parties in the USA or elsewhere?

Does anybody have knowledge and/or references of Other bug reports being collected by third parties in the USA or elsewhere?

Philip HAugust 1, 2017 10:13 AM

Microsoft may have completely different incentives in general, but they have some pretty smart people using WER and other data to find and fix nasty exploits. This blog post from MSRC impressed me to no end when it was posted in 2015: https://blogs.technet.microsoft.com/johnla/2015/09/26/the-inside-story-behind-ms08-067/

This more recent post highlights a similar doggedness based on nothing more than a Twitter boast: https://blogs.technet.microsoft.com/srd/2017/06/20/tales-from-the-msrc-from-pixels-to-poc/

CowardAugust 1, 2017 10:18 AM

When I worked at Microsoft I discovered that the company was curating and feeding NSA this information. This is one of many such internal programs.

Glad to see Der Speigel reporting on it.

Adam SAugust 1, 2017 10:29 AM

Also, this "Microsoft won't have the incentive to examine and fix problems until they happen broadly among its user base." is not the case. Getting vulns early so that there's time to test patches is very helpful. At least when I was there, there was a desire to find and fix bugs as deeply as possible. Design flaws, like autorun, much more difficult to fix because fixes break compatibility, and when you break compatibility, you risk people avoiding updates. There was plenty of energy available for examination, the trick was turning it into fixing.

JohnnySAugust 1, 2017 10:34 AM

You get enough people banging on keyboards generating errors, sooner or later they will trip over a flaw that can be exploited. By collecting the reports, you've got access to the biggest fuzzer in the world: The entire Internet!

CowardAugust 1, 2017 10:49 AM

@Adam S

When I was there, backcompat was a priority to bug fixes, but bug fixes were taken very seriously. Except for programs where the goal was to create implantable software/backdoors or persist existing ones for intelligence.

If an intelligence backdoor was found independently there was an entire process by which we would notify, process and sunset their access via that vulnerability in a timeline where they would have a replacement capability. Luckily I didn't have to deal with any of the politics.

CowardAugust 1, 2017 10:51 AM

@Garrett B

Microsoft sends it all to NSA willingly in exchange for a contract relationship.

no such acrobaticsAugust 1, 2017 11:18 AM

"Think the NSA issued themselves a *.microsoft.com cert?"

They have the keys to the kingdom. A cert is just a hall pass.

nerd school academyAugust 1, 2017 11:32 AM

"telemetry that is flowed back to the mothership over https."

Are you seemingly thinking the https handshake stops the NSA from intercepting it? Lol.

Nefarious Shady Associates August 1, 2017 11:50 AM

"This data is very helpful for finding zero-days."

Which aren't patched until people other than the NSA know about them historically,
and are used meanwhile, historically, by contractors for same.

"the telemetry channels became fully encrypted via TLS"

OMGerds, the fabled transport layer security? That ought to keep pesky NSA *hats out.

https://www.theguardian.com/world/interactive/2013/nov/01/snowden-nsa-files-surveillance-revelations-decoded#section/3

Unless they own the entire world and merely tolerate your futzing around in it for the analytical data your habits provide for extrapolative/exploitative enterprises, that is.

JimBoAugust 1, 2017 12:14 PM

The article seems to have an inaccuracy saying that every time a crash occurs Microsoft is notified.

“intelligence agents are then automatically notified any time the operating system of that computer crashes and its user receives the prompt to report the problem to Microsoft.”

The notification only occurs when the user agrees to send the report. A very small portion of the users will ever send the data. This is valuable for finding exploits, but not for obtaining personal information. Do we really think criminals and terrorist will send the report to Microsoft?

David HessAugust 1, 2017 1:11 PM

No detectable certificate shenanigans would be necessary if the NSA has a copy of Microsoft's private key(s).

Julian AssangeAugust 1, 2017 2:26 PM

This scenario was a condition of Microsoft's parole in the AntiTrust case (lodge by Al Gore and Co).

They kicked Bill out of the computer industry. He left.

NSA now owns MS to see how well their BS hackware works against you.

ab praeceptisAugust 1, 2017 3:35 PM

My knowledge of windows is very limited but: I think it's not all that innocent. After all (I guess) a windows box with problems sends out quite some information which, particularly when collected from many boxen might come to be quite an attack surface.

And I agree with Bruce Schneier. Quite probably those boxen send out i.a. their exact version (update status), so someone with access to that quite probably can find some correlations and effectively get a map of boxen and vulnerabilities.

But I see another aspect, too: I presume that a rather high proportion of those boxen belongs to, Pardon me, not exactly bright bulbs and/or "he, who does no wrong, can be transparent without worry" guys. This might actually be used more as a white list, i.e. people whom not to eavesdrop on.

Chet AmericanmanAugust 1, 2017 4:07 PM

@Coward, regarding the contract you mention, do you have an RMS TAT number and MIPR ID for it? Or are you saying that unrelated contracts are issued quid pro quo for betraying Microsoft customers with unfixed vulns?

Chet AmericanmanAugust 1, 2017 5:29 PM

Thank you, good to know. Then this approach will be a helpful precedent for holding Microsoft accountable in foreign jurisdictions. In the same way that Cisco helped China, Microsoft acquiesced in selection of victims for JSOC death-squad killings and CIA torture. Microsoft helped NSA paint bulleyes on the backs of journalists, human rights workers, and other protected persons.

https://www.eff.org/press/releases/eff-court-cisco-must-be-held-accountable-aiding-chinas-human-rights-abuses

Clive RobinsonAugust 1, 2017 7:26 PM

@ vas pup,

... could AI be utilized to analyze and separate fake network/servers/honey pots and real network?

The big problem with a honeynet is tp do it properly it needs to be a real network with real computers doing real work. Otherwise it can be detected by some alarmingly simple methods.

For example many honeynets are not real physical networks of independent machines. They are actually only one or two computers runing VM's that pretend to be multiple computers. The problem with this is that such arrangements have a weakness, they all share the same hardware... which means you can develop tests to detect common components. I mentioned this on this site years ago, but when running multiple VMs on one morherboard it has a single crystal to drive the clock generation for the whole morherboard. Thus all the VMs have clock signitures exactly thr same. The crystal will drift up and down in frequency due to things like external temprature and changes in the CPU load. But unlike real independent computers all the VMs will remain in perfect lockstep timing wise. Thus the simple use of a basic script kiddy enumeration (ping) attack will revel the VMs keeping time synchronisum, from ordinary networks with multiple independent machines thus crystals thus timing information .

WaelAugust 1, 2017 11:29 PM

how valuable would this database be for finding new zero-day Windows vulnerabilities to exploit?

Rhetoric question. Most valuable.

Non Secure ArgumentAugust 1, 2017 11:43 PM

"A very small portion of the users will ever send the data."

That's a completely fallacious assumption you just asserted as a fact. #Trump_skills

Just about every single non-enterprise W32 end-user "noob" I ever come across has basic telemetry and error reporting turned ON, as it is ON by default in EVERY Win distro.

JUST the OS itself, before you add all the OEM 3rd party pingers, Lenovo Superfish et al.
Lately in 10 it's trending to MORE TELEMETRY, not less. On by default. Everywhere.

SOME people are aware of it and turn off / opt-out in the settings, but A LARGE % of these people DO NOT KNOW THESE SETTINGS EXIST.

I'd be surprised if the "small portion" were not hovering around 50% of home users.
But I'll make clear that I'm completely guessing based on my own anecdotal evidence.

#Rather than stating that as fact.

Clive RobinsonAugust 2, 2017 12:56 AM

@ Bruce,

I don't remember this being discussed back in 2013.

Possibly because this was not the first time such NSA activity was known about.

Have a look back at the CarrierIQ incident two years prior to that.

The CarrierIQ "test and support" software had been covertly installed on Mobile Phones by US mobile operators. It was allegedly discovered by a researcher seeing odd data being sent from their phone.

However it was fairly widely known about in the Mobile Phone Industry, and the possible depth of it's penetration / intrusion into Smart Phones was known to "help desk staff" who had a high turnover (as many were also expected to act as sales staff or get the boot).

Although to be fair it was not widely known how the data was sent. Most either did not think about it or had assumed due to lack of familiarity with such things, it was "secure". When infact it was anything but, it was sent effectively as plaintext across the internet to CarrierIQ's servers, as much "test harness data" still is, and will continue to do so for the foreseable future. Because it's seen as "test data" not PII / legaly protected communications etc.

It was indicated at the time that it was "quite likely" the SigInt agencies were aware of it and that the NSA for other 5eye members were reading it off the Internet from a point just upstream of the CarrierIQ internet gateway for their service.

Back then people who knew or who had a very good idea of what was going on because they had seen it before would say "quite possibly" or "quite likely". This was to avoid getting the then "Conspiracy theory nut" backlash that was the normal knee jerk reaction, or even a visit from the likes of the FBI.

You may remember a stand up commedian got quite upset about it, and as Al Franken was at that time the Democratic Senator for Minnesota, he had some clout to get answers. Unfortunatly others who were well aware of it such as Democratic Senator Dianne Feinstein from California said and did nothing, even though she was Chair of the Senate Intelligence Committee, and should have been aware of it.

Thus Al Franken got pointed in the wrong direction and little or nothing actualy got done about it (another aspect that people should by now be familiar with).

When Ed Snowden became known and the size of the trove of documents likewise it was hopped that this sort of thing would be made public. But due to the way the trove is being released it may never happen.

Any way CarrierIQ got swept away from the public gaze as "old news" fairly quickly and few made note of the fact it was a test harness failing. Which is a shame because it is exactly this sort of test harness reporting over the Internet that the SigInt agencies dream of. Because they just have to sit and watch the traffic go by hovering it all up.

It is unknown if we will ever know if the NSA had a direct input into CarrerIQ's decision not to encrypt the data, but it is unlikely for a couple of reasons. Firstly as we now know they primarily work at arms length through other agencies where possible because it gives both sides deniability (hence the FBI and NSLs). Secondly nearly all such technical support harnesses are effectively plaintext when they "call home" as is the MicroSoft system and many others (also think Smart Meters, Bio-medical equipment and our evere present new IoT friends).

It was said again back in 2011 that such test harnesses make ideal security end runs. But I guess few were listening as the US were still in the "We don't do that" mentality (even though told repeatedly for half a century that they do trough others).

It's these sorts of test harnesses and more importantly the hooks at low level in the OS's that should make it clear to the most brain dead why there is no security on our PC's and Mobile Phones. Because they extend the communications end point beyond that of the security end point. Thus "end run" the security application no matter how good it is or by whom it has been designed or built.

This problem with end point security and test harness hooks in the OS does get pointed out from time to time. But most people including many technical security people appear to have a "blind spot" with regards technical support test harnesses even though they do come up in a quite nasty way from time to time.

Such as those on telephone exchange equipment to spy on the Greek Government mobile phones in 2004/5 whilst the Athens Olympics were on. The fall out of which resulted in a suicide/murder of atleast one person, and came to the public attention in 2007.

As they say "All you have to do is, keep your eyes open / join the dots".

matteoAugust 2, 2017 2:08 AM

i have disabled them.
they might send back *crash dumps* that contain memory
and memory might contain passwords (especially for firefox, thunderbird...)

Who?August 2, 2017 5:50 AM

When will we accept that U.S. corporations and the intelligence community are in the same boat?

mostly harmfulAugust 2, 2017 6:10 AM

@Clive writes:

This problem with end point security and test harness hooks in the OS does get pointed out from time to time. But most people including many technical security people appear to have a "blind spot" with regards technical support test harnesses even though they do come up in a quite nasty way from time to time.

Such as those on telephone exchange equipment to spy on the Greek Government mobile phones in 2004/5 whilst the Athens Olympics were on. The fall out of which resulted in a suicide/murder of atleast one person, and came to the public attention in 2007.

James Bamford's 2015 article in The Intercept, regarding that episode in Greece: Did a Rogue NSA Operation Cause the Death of a Greek Telecom Employee? https://theintercept.com/2015/09/28/death-athens-rogue-nsa-operation/

Block OutboundAugust 2, 2017 6:39 AM

Besides error reports, MS uploads telemetry data from every user thousands of times per day. Certainly, some of it is intentionally or unintentionally "shared". As said elswhere, there are no settings to turn it all off.

One thing that works well against outbound "telemetry" is a solid firewall that blocks ***outbound packets***. Even Windows firewall can do that. Also, CCleaner and Bleachbit will erase a lot of the logs, hopefully before they are sent to the mother ship.

It's very disappointing to see how closely the government and corporations work against the privacy and security of honest citizen users.

Who?August 2, 2017 6:48 AM

@Block Outbound

If we believe what @Coward says then it is easy to spot that telemetry data is intentionally shared with the NSA turning Windows into the most dangerous product ever developed for privacy and security.

Dirk PraetAugust 2, 2017 6:52 AM

@ Non Secure Argument

SOME people are aware of it and turn off / opt-out in the settings, but A LARGE % of these people DO NOT KNOW THESE SETTINGS EXIST.

Exactly. Which makes things even worse is that it is also far from evident for those who do know and do care to turn these settings off. There are however a couple of free utilities out there that can disable known spying features (including the crash dumps) through an easy-to-use GUI. I personally use O&O ShutUp10 and SpyBot Anti-Beacon. Keep these utilities up to date and re-run them after every Windows Update since some of these updates covertly reset to default certain settings.

JUST the OS itself, before you add all the OEM 3rd party pingers, Lenovo Superfish et al.

Almost every stock Windows distribution on a newly purchased PC comes with a staggering amount of vendor-provided cr*pware that provides little to no added value, phones home and installs itself as start-up programs making the boot process painstakingly slow even on machines with powerful CPU's and plenty of RAM. Whilst it is ill-advised for the average user to manually remove those from the Windows registry (see overview of all autostart stuff here; it's unbelievable), a utility like CCleaner offers an easy way to either remove or disable those. As a rule of thumb, just disable anything that doesn't have "Microsoft" in its description.

Lenovo Superfish and other persistent malware are in a highly insidious category of their own since they use the Microsoft Windows Platform Binary Table (WPBT) to (re)install themselves from EFI firmware. Whereas most traditional antivirus software will eventually detect them , they often require dedicated utilities or even a full firmware update and OS reinstallation to get rid off.

Another useful tool in the arsenal is Malwarebytes (also available for Mac), which in my experience also picks up potentially unwanted programs (PUP) that traditional virus scanners often overlook. Be advised however that this is a serious memory hog that especially in combination with another TSR virus scanner may slow down your machine to a crawl. I usually recommend to execute it manually instead of having itself start up automatically with Windows.

Since I believe that most, if not all US antivirus vendors are also in bed with the US IC, it makes sense for users concerned with more traditional 5Eyes sponsored malware to replace the likes of McAfee and Symantec with Kaspersky. Last but not least, knowledgeable users may benefit from installing Binisoft's Windows Firewall Control (WFC), a slick and inexpensive utility that integrates with Windows's native firewall to alert on unknown outgoing connections.

CAVEAT: This is far from an exhaustive list on how to secure Windows, and none of the tools above will shield you from targeted corporate or state actor spying. They are tools to mitigate a number of common threats to Windows users. No more, no less. If your security, anonymity and privacy is of genuine concern to you, look for more comprehensive guidelines on-line or realise you are on the wrong platform to begin with.

@ ab praeceptis

This might actually be used more as a white list, i.e. people whom not to eavesdrop on.

In what psychedelic parallel universe does it even remotely make sense for an IC to use highly controversial methods, piggybacking on an already exposed and compromised "special partner" to create a white list of "innocent idiots" ? This is just another abusive tool to search for vulnerabilities on the back of ignorant users that have never given any consent to do so.

The only appropriate reaction for anyone is to turn it off, plain and simple, and even the mere suggestion that continuing to send those crash dumps could put them on a no-spy list is by far the dumbest piece of advice I have ever heard since we first started discussing Snowden on this blog.

vas pupAugust 2, 2017 8:30 AM

@Clive - Thank you for input very good as usual.
Clive, you'll like this article:
http://www.bbc.com/future/story/20170801-the-ghostly-radio-station-that-no-one-claims-to-run
“MDZhB” has been broadcasting since 1982. No one knows why.
“Enter the “numbers stations” – radio stations that broadcast coded messages to spies all over the world. Soon even the British were doing it: if you can’t beat them, join ‘em, as they say. It’s quite difficult to generate a completely random number because a system for doing so will, by its very nature, be predictable – exactly what you’re trying to avoid. Instead officers in London found an ingenious solution.
[!]They’d hang a microphone out of the window on Oxford Street and record the traffic. “There might be a bus beeping at the same time as a policeman shouting. The sound is unique, it will never happen again,” says Stupples. Then they’d convert this into a random code.
It also fits with a series of arrests across the United States back in 2010. The FBI announced that it had broken up a “long term, deep cover” network of Russian agents, who were said to have received their instructions via coded messages on shortwave radio – specifically 7887 kHz.
[!!!]It may come as a surprise that numbers stations are still in use – but they hold one major advantage. Though it’s possible to guess who is broadcasting, anyone can listen to the messages – so you don’t know who they are being sent to. Mobile phones and the internet may be quicker, but open a text or email from a known intelligence agency and you could be rumbled."

Pagers (initially) fit the same pattern above - you never know where is addressee of transmission located.

Regarding that station, my guess would be that multiple listeners could be programmed (used post-hypnotic technique without even their awareness)to activate particular pattern of activity when on that frequency is broadcasted code message which could be not word or text, but rather audio (possible subliminal) activation 'key' (pattern)meaning listener could not override the 'key' by logic/will since it is subliminal. But, I am thinking could this frequency be jammed altogether (like Soviets jammed BBC, Voice of America transmissions) or by using EMP?


ab praeceptisAugust 2, 2017 10:03 AM

Dirk Praet

In what psychedelic parallel universe does it even remotely make sense for an IC to use highly controversial methods, piggybacking on an already exposed and compromised "special partner" to create a white list of "innocent idiots" ?

For instance, in a universe where spooks need "reliable" bots. Just think of a russian or a chinese idiots windows box and of "we can make attacks look like coming from an "origin" of our desire".

Or in a universe where nato generals and politicians increasingly open talk about cyber warfare.

Which, oopsie, happens to be *our* universe,

Noob Stick AutobuyAugust 2, 2017 11:20 AM

" the NSA turning Windows into the most dangerous product ever developed for privacy and security. "

I think MS gets SOME of the credit, anyhow...

Dirk PraetAugust 2, 2017 11:33 AM

@ ab praeceptis

For instance, in a universe where spooks need "reliable" bots.

You're just making things up as you go. There's far more efficient ways to create a bot army than examining M/S crash dumps. Try using Occam some time.

You may have noticed that I usually don't comment when folks like @Clive, @Nick P, @Wael or even yourself are talking about stuff that's far out of my comfort zone. Then I just try to understand and learn. Perhaps you could consider the same.

Clive RobinsonAugust 2, 2017 11:38 AM

@ vas pup,

Though it’s possible to guess who is broadcasting, anyone can listen to the messages – so you don’t know who they are being sent to.

The BBC has actually got that wrong, and the joirnalist should know better, if he's actually talking to people who know the history of the subject.

There are three basic types of radio the Tuned Radio Frequency or TRF recevier that uses an on frequency oscillator that is critically damped. This makes it behave like a very high gain amplitude tuned to a single frequency.

A Direct Conversion receiver that uses an on frequency oscillator phase shifter and a couple of mixers. The DC output of the mixers is then fed into another phase shift network to recover the modulation

Both of these produce an "on frequency" signal that can be picked up by suitable equipment as it frequently goes up the antenna like a very low power transmitter. It can often be picked up by air bourn equipment at three or four miles distance.

The third type of receiver is a hetrodyn or more commonly Super Het. It uses an off frequency oscillator to mix --hetrodyn-- the RF input frequency to an Intermediate frequency (IF). Due to the process of mixing you actually get the RF and Local Oscilator (LO) frequencies at the two mixer inputs, and the IF out is actually two frequencies the sum and difference of the RF and LO frequencies. Due to the poor nature of low cost receivers not only the LO frequency but the two IF frequencies get radiated.

It was this property that some Telivison Detector Equipment used as the old valve mixers and early transistor mixers radiated the IF back up the screen of the antenna coax rather well. So much so you could usually hear the third harmonic of the 33-37MHz IF on a VHF broadcast receiver (which I did when a preteen and an old valve receiver I bought for pennies at a jumble sale).

Whilst it's a lot better since the introduction of the EMC regs in Europe and other countries playing catch up these signals can still be heard at quite large distances. Thus the old High Frequency Direction Finding (HFDF or Huf Duf) can be used to find unwary agents of a foreign power.

For those with a copy of SpyCatcher you can read of the adventures of the late Tony Sale --of Bletchly famr-- in hunting down Russian illegals by aircraft and van.

As for the actuall transmitter it's most probably just "station keeping" however it may be of a very high stability in which case it could be used as a "tuning aid" for people with non synthesized radios. It could also work with other transmitters and difference modulation to send slow signals that can only be detected in certain locations (look up MIMO systems to see why).

WaelAugust 2, 2017 12:04 PM

@Clive Robinson,

For those with a copy of SpyCatcher you can read of the adventures of the late

Ultra bizarre! RF, IF, Superhet, and Spycatcher too?

Keep reminding me that I got the wrong book. Hard to stay pleasant these days!

@Dirk Praet,

Then I just try to understand and learn.

I do the same. Either that or I ignore the discussion. Not interested which programming language is "more secure", for example. I said my piece a while back, and that's the end of it. Moot discussion, unless one is actively engaged in the design of such language. I usually ignore compiler related discussions -- not my cup of tea! I hardly ever read them.

ab praeceptisAugust 2, 2017 12:23 PM

Dirk Praet

"There's far more efficient ways to create a bot army than examining M/S crash dumps."

Like? Please, elaborate.


"You may have noticed that I usually don't comment when ... stuff that's far out of my comfort zone."

Is that so? This very post of yours (and many others) suggest otherwise.


Let me help you. People who have those problems reports sent to microsoft are either clueless (and quite probably careless) or bound by some organizational standard. Moreover they are unaware and/or ignorant of security. Plus, by definition they run an utterly insecure "OS".

In other words: An ideal group for spooks (and criminals, if one wishes to differentiate).

Now, look at typical botnets for comparison: The common factor there is typically "has xyz hardware rev. <= a.b.c". That (or more precisely the technical implication behind that) is enough to bring those devices into a botnet but that's pretty much it.
With the error report win machines, however, a lot more is known and even much more can be found by e.g. correlating (e.g. auto updater vs. lazy occasional updater, country, language, static IP or dhcp'd etc).

Dirk PraetAugust 2, 2017 12:45 PM

@ ab praeceptis

Is that so? This very post of yours (and many others) suggest otherwise.

I'm done with you. You're talking out of your *ss and there's no point anyway in trying to have a civil discussion with an absolute git who has been put on notice before for breaching forum etiquette.

If this is the new standard of this forum, then I'm out of here. I'm gonna go see Patti Smith. Have a nice day spouting your insults and nonsense to someone else.

Clive RobinsonAugust 2, 2017 12:49 PM

@ Wael,

not my cup of tea! I hardly ever read them.

Hmm a Romany Madam does not read her own tea leaves... Though what you would look like in "Gypsy Costume" heaven alone knows, I hope ;-)

JG4August 2, 2017 1:10 PM


@Clive

"Both of these produce an "on frequency" signal that can be picked up by suitable equipment as it frequently goes up the antenna like a very low power transmitter. It can often be picked up by air bourn equipment at three or four miles distance," and "Due to the poor nature of low cost receivers not only the LO frequency but the two IF frequencies get radiated."

the Greek resistance had the same problem with local oscillators getting picked up by the occupation forces. pretty much everyone on your planet who has fallen out of favor with the authorities is susceptible to the same problem. today's RF environment probably has a lot more clutter from the PCs and IoT.

perhaps a type of data diode could help with the radio receiver case. there should be a way to take the incoming RF off the antenna connector, amplify it and directly drive an optical transducer, e.g., laser diode, with the resulting signal. I am assuming sufficient bandwidth in the optical transducer, which easily to up to 40 GHz. now the optical signal is passed via single-mode fiber into a robust Faraday enclosure, including appropriate magnetic shielding, for signal processing. it should be possible to build a receiver with essentially zero electromagnetic emission.

I introduced yesterday the concept of an audio data diode that uses some flavor of pseudorandom white noise as a carrier. actually, a pair of them to couple secured content in and out of an insecure cell phone. there are a lot of useful elaborations on what can be done with cell phones using the basic building blocks that I laid out. the ones that don't tamper with the RF will be immune to FCC rule changes that can be expected if there is any meaningful adoption.

I've been meaning to propose a method for detecting CCDs. you've mentioned before the red-eye effect, which will occur in all cases where there is a lens. there may be effective countermeasures that can be built into the lens and/or CCD assembly against this detection method. it should be possible to promote charge carriers in any semiconductor camera/image sensor with a laser flash. this will create a population of charge carriers that are susceptible to probing with nonlinear effects. silicon is not a direct bandgap semiconductor, but it may still be possible to see the excited state with a second laser pulse, and measure the resulting laser-induced fluorescence. this will be of great interest to DoD for detecting drones and other battlefield robots.


WaelAugust 2, 2017 1:12 PM

@Clive Robinson,

Hmm a Romany Madam does not read her own tea leaves

Clever :)

Though what you would look like in "Gypsy Costume"

Not half as bad as a Two meter bearded Klingon (with or without custom) ;)

heaven alone knows

Heaven, in your case, is a plural (a plural of many, in other languages.) Remove the 's', and you'd be more consistent :)

ab praeceptisAugust 2, 2017 1:17 PM

Dirk Praet

Two remarks are sufficient.

a) as you try again to smear me as "well known and already warned off evildoer" I'll add the part you repeatedly "forgot" to mention: our host apologized to me.

b) just look at our posts. In mine you'll find arguments; one may feel that they should be discussed but they are there and good-faith efforts. True, I'm not always soft-spoken but I deliver arguments for my position.
You, however, (in this case anyway) are quite short on arguments and rather prefer wanton assertions and smearing and personally attacking me (and I even don't complain. I simply stick to the matter).

Just look at this very situation. You asserted something (painting me as clueless), I questioned it and asked you to elaborate ... and now you make a big scene and threaten to leave instead of simply delivering arguments for your assertion.

WaelAugust 2, 2017 1:32 PM

@Dirk Praet,

If this is the new standard of this forum, then I'm out of here

Don't do that, mate!

ModeratorAugust 2, 2017 3:41 PM

@Dirk Praet, @ab praeceptis

"If this is the new standard of this forum, then I'm out of here."

It is not. Disengagement is a good idea.


ab praeceptisAugust 2, 2017 4:39 PM

Dirk Praet

Although you repeatedly even insulted me (e.g. "talking out of your *ss", "absolute git") I stayed cool and stuck to the matter.

While I find your "I'm leaving!" pathetic and seem to even spot an undertone of blackmailing, particularly as you have tried more than once to attack me using social mechanisms,

I want to clearly state that I have no hostile feelings whatsoever towards you, nor do I have any desire to see you leaving or even just silent.

I'd be surprised if the two of us became friends but I stretch out my hand to you.

AforAugust 2, 2017 5:26 PM

re C. Robinson, "The CarrierIQ "test and support" software had been covertly installed on Mobile Phones by US mobile operators."

by... mobile operator staff suborned as agents by CIA. There, fixed it for ya.

https://theintercept.com/2015/09/28/death-athens-rogue-nsa-operation/

This incredibly obvious industry truism got classified ECI. With a straight face! The researcher who found CarrierIQ is lucky he didn't get suicided like Costas Tsalikidis. Langley has two more CIA fugitive murderers to harbor: William George Basil and Richard Eric Pound.

WhiskersInMenloAugust 2, 2017 6:29 PM

Perhaps a silver lining.
The NSA has a charter to protect the US and in that roll they should be part of the defect reporting and fixing process.

WindowZ machines are used by our government and also most other companies and governments. Some flaws are easy to track and occasionally use. Other flaws allow an attacker invisibility and some security in their abuse.

My personal bias is all bugs should be vanquished if only to reduce the signal to noise ratio of problem reports.

Another problem is features that used in foolish ways cause problems. Telnet and anonymous ftp are features that are not bugs but foolish to use except when needed.

So the never say anything folk can help by saying the right things to vendors.
The FBI charter seems to mandate bug reports to vendors. The CIA needs to be reminded that bug exploits cut both ways. The apparent power that an exploit might appear to hold is also a vulnerability that might put the nation at risk.

I believe the Obama directive to report bugs is still in effect.

RachelAugust 3, 2017 12:54 AM

Though what you would look like in "Gypsy Costume" heaven alone knows, I hope ;-)

I don't know what Wael would look like either. But I sure as hell know he'd be singing Habibi!! With perfect pronunciation

WaelAugust 3, 2017 2:14 AM

@Rachel, @Clive Robinson,

But I sure as hell know he'd be singing Habibi!! With perfect pronunciation

You're half right. I can pronounce it very well, but I don't sing. I don't even know the lyrics.

Hmm a Romany Madam does not read her own tea leaves

Of course not! She'll read Bob's tea leaves! Speaking of Tasseography:

Cryptography and Steganography are old hats. If Alice and Bob are halfway decent Tasseographists and they dump the leaves after one message exchange, then Eve will be left helpless. PQC secure, too. Crash dumps are only good for compost and they leak no readable information. No communication channel is required either! But error correction is a female k9.

Dirk PraetAugust 3, 2017 4:47 AM

@ Moderator

Disengagement is a good idea.

I will take your advice, refrain from any further engaging this person and join those who already do. Whether or not his rather personal style of discourse adheres to this here forum's etiquette I leave up to you and our host to judge.

lurkerAugust 3, 2017 8:29 AM

Microsoft actually tried to sell a service "Security Error Reporting and Analysis (SERA). Malware Detection and Vulnerability Assessment by Analyzing Windows Error Reports."

A corporation would configure their Windows machines to send crash dumps, error reports etc to a central machine. Then they'd run a set of data mining tools against those crashes to try and spot malware/exploit attempts.

albertAugust 3, 2017 11:23 AM

@vas pup, @Clive,

It's been a long time since I thought about RF receiver emanations, and yet what doesn't use RF nowadays? I wonder if the software-defined radio ICs are prone to that?

BTW, an interesting document to peruse is:
(FCC table of frequency allocations)
https://transition.fcc.gov/oet/spectrum/table/fcctable.pdf

For the EU:
http://www.onlineconversion.com/downloads/european_frequency_allocations.pdf

No allocations below 8300Hz (9000Hz in the EU). Lots of cheap, powerful audio amplifiers around. Do the math.

. .. . .. --- ....

Clive RobinsonAugust 3, 2017 11:54 AM

@ Albert,

No allocations below 8300Hz (9000Hz in the EU). Lots of cheap, powerful audio amplifiers around. Do the math.

I have, the Ham record holder is a friend of mine.

The problem is antena efficiency / ERP and it's very very dire.

Clive RobinsonAugust 3, 2017 3:09 PM

@ Albert,

If you want a go at VLF Dave Bowman has a web site with some kits, ccts and instructions. My prefered option is a Class F Amp for it's high efficiency, but you still have to push that up a wire with a wavelength being 30,000m at 10KHz youl'd be looking at having a county sized back garden or gamma loop matching into a prairy farmers fence.

You mention you sometimes have younger family members over, have a look at the coke bottle rocket using methanol etc for fuel. It's a lot easier and cheaper than water rockets and more exciting for the kids.

As Dave demos it he's a little safety conscious. I probably would not do it any longer but I used to drill the hole in the center of fhe bottom and add thin carboard fins and due to the bottom of the bottle being dimpled 5ml of meths gets in there without running out. And we used to launch them "Indian style" that is by hand which I urge you not to try, as getting the timing right can take the hair of the back of your hands. Oh and I've had fun with 2lt bottles that way as well.

I won't go into the physics of it but if you look up "jam jar jet engines" that are effectively a pulse jet you will find something like,

http://www.instructables.com/id/Be-an-Aeronautical-Scientist%3A-Make-your-Own-Jet-En/

You can do the same with a plastic coke bottle, but it will only work once or twice as the burn heat will deform or melt the bottle causing it to fail, occasionaly explosively. However when you know the right size for the hole then you don't care. Just be quite carefull, remember the Jet Propulsion Lab started because a couple of students nearly burnt their dorm down.

Who?August 4, 2017 5:31 AM

@ Noob Stick Autobuy

[citing me] "the NSA turning Windows into the most dangerous product ever developed for privacy and security."

I am not blaming the NSA, I am blaming Microsoft. Please, read the full phrase!

If we believe what @Coward says then it is easy to spot that telemetry data is intentionally shared [by Microsoft] with the NSA turning Windows into the most dangerous product ever developed for privacy and security.

Microsoft is turning its operating system into the most dangerous surveillance platform at the service of the NSA and, perhaps, other governments around the world.

Non Sum AgreementAugust 4, 2017 7:04 PM

Having seen it in action again, I love this moderation style. It's among the best.

That subtle and forgiving human touch that most humans appreciate. -TM

Bruce for President? Who said that?

RobertTAugust 5, 2017 2:46 AM

Long time since I graced these pages hope I'm still welcome.
special shout out to NickP and CliveR

Dirk PraetAugust 5, 2017 5:50 AM

@ RobertT

... hope I'm still welcome.

If that's really you, most definitely so! I don't think I'm the only person here to have missed your insights. Now if we could only get @MikeTheGoat back too 8-)

Clive RobinsonAugust 5, 2017 7:28 AM

@ RobertT,

Nice to here from you, NickP appears to be migrating over to Hacker News and thrn Lobsters as certain types became more evident in the run up to and post US elections.

I hope you are well and intact and as the used to say "found a nicer hole" in what is fast becoming a FiveEyes -v- The rest of the world Cyber-saber rattling and Cyber-Drum banging lead up to kinetic action...

@ Wael,

Yes MiketheGoat has not put in a reappearance I hope if he is lurking he is well and just busy.

albertAugust 5, 2017 10:54 AM

@Clive,

Here in the colonies, we have lots of wide open spaces. We used to run our long wires between trees. Hide the insulators in the leaves and they're almost invisible. With new materials, you might create huge antenna systems. Unbeatable for receiving signals. As for transmitting, well, the barrel-sized coils I've seen for impedance matching give new meaning to the term 'tank circuit':)

. .. . .. --- ....

RobertTAugust 5, 2017 6:03 PM

@Dick and Clive thanks for the welcome back, not sure how I can possibly prove who I really am so I'll have to let others make up their minds. That said, I have to avoid discussing any highly technical aspects of cryptanalysis, freedom both personal and national comes at a cost.

As for Five-eyes, yeah that's business as usual and it's not what worries me, there's nothing that has been revealed over the past few years that I didn't already know or have very good reason to suspect. IMHO True Innovation in the security space is no longer coming from through the traditional 5-eyes route and that's both exciting and worrying at the same time.

Nick PAugust 5, 2017 8:22 PM

@ RobertT

You're alive! And back! Awesome! :)

I didn't waste the things you taught me about hardware. I went on to create an unpublished flow for securely creating it settling on something trace-based w/ untrusted producers but trusted verifiers so one could leverage commercial components that nobody was going to be able to re-implement. I did find some FOSS stuff like QFlow, though, that might be made to work on older nodes. I had to stop at analog and RF as they were too manual and detailed to cheat with my 1,000ft view style. I'll have to recommend using diverse specialists if a FOSS effort takes off. Most resources for analog are either theory-heavy or blind hands-on. Best I found for learning it later was

I posted a summary of hardware subversion issues here and a smartphone analysis integrating some of your claims here. Collected FOSS stuff for FPGA's and synthesis here. Came up with the bootstrapping idea of using 0.35 micron or higher as initial node since, from what I read, you can still photo it in a microscope that doesn't use electrons. Maybe do visual models of expected layout, tear down a sample of them, and compare against visual model. Each user tears down some & uses others randomly w/ maybe computer vision automating the comparison at some point. The early ones are used in combination with trustworthy, easy-to-learn FOSS to produce or check the next set of stuff. And on and on.

Far as bootstrapping trust, rain1 and I have collected about every simple technique of constructing interpreters, hardware, or compilers we can find in one place (currently content over presentation). I just acquired the NAND2Tetris book with idea of slowly learning hardware up to basic CPU design I can then use to verify Wirth's RISC CPU he ported Oberon to. Then, I'd just have to create or port the low-level build system (esp linker and assembler) to my version of it. Then, get the initial bootstrap done by hand-porting Oberon source (esp SYSTEM, I/O, and compiler) to direct assembly or a macro-assembler. I'd then have a clean image of a well-documented system running on a CPU I understood possibly on an ASIC in a shuttle run. However, still need a specialist to do PHY's or mixed-signal parts but that's as easy as finding a retired EE that doesn't have a reason to backdoor your stuff. ;)

I also tried to condense several decades of research on software assurance techniques and worked examples into just a few pages that new people could follow without getting overwhelmed. Put that here on a very-low-noise forum with lots of hands-on technical folks. Well-received. Otherwise, been on Hacker News and Lobsters mainly talking to specialists in different fields teaching, learning, and especially calling bullshit on echo chambers. It was neat meeting John Nagle of Nagle's Algorithm, Alan Kay, and an EE who came up with a secure-trusted/fast-untrusted ASIC scheme similar to my own in published journals. Fun times. I keep a list of at least some of the essays I put there but many wouldn't be new to Schneier readers as I developed them here.

Feel free to comment on anything positive or idiotic about my hardware-related stuff. That will confirm or reject who you are pretty quickly as we'll remember your patterns. ;)

Nick PAugust 5, 2017 8:33 PM

@ RobertT

While we're at it, I've found that few EE's seem interested in open hardware. The number of volunteers willing to contribute is always low. It just seems much lower for hardware engineers. Got any thoughts on that outside maybe the risk of patent or trade secret issues from prior work leaking into implementations? That was one of my guesses. Other being you people are way too practical for the idealistic needs of the world. :P

As I thought on it, I figured the competitiveness of hardware meant that we couldn't build replacements for the stuff dominating industry. We might be able to build it into appliances, thin clients, embedded systems, and so on a piece at a time. Eventually having, say, a secure SoC with RISC-V and all the peripherals open. I had two ideas for that:

(a) Get academics to build as much as possible like they already do but open-sourcing implementations done on a common node. Maybe 45nm SOI that Rocket Chip Generator is already on. We need DDR, PCI, USB 2/3, and so on not more CPU's. Well-tested implementations on real nodes that are available cheap to free can be picked up by integrators building the risky projects at *much less* cost.

(b) Use Shenzhen which seems idea for developing cheap hardware. They were first thing that came to mind when people wanted an iPhone replacement with open firmware, button to disable radio, and so on. Likewise, with so much hardware outsourced to Chinese, I'm sure there's people there that could develop and verify the hardware at low price with people of other nationalities just checking it all. There's still tool or fab subversion to consider but might eliminate lock-in and vast majority of attackers. That's still worth something. As before, it's done piece by piece with sales of each piece funding the next or as a loss leader by one or more profitable companies like with RISC-V work. They pay for a secure service or server with the expensive CPU/board it runs on being a bonus and differentiator offered at unit cost.

(c) Get a FPGA done on 28nm or most cutting-edge node possible as you advised before. Target commercial (i.e. Mentor Precision) and/or open tooling (i.e. QFlow) to it. From there, do security through obfuscation in and around the image running on it. The actual FPGA's might be torn down by Chipworks or something periodically with extra supply chain security.

Your thoughts?

Clive RobinsonAugust 5, 2017 8:59 PM

@ RobertT,

IMHO True Innovation in the security space is no longer coming from through the traditional 5-eyes route and that's both exciting and worrying at the same time.

History shows that sudden wealth is seldom good.

Spain stole gold and silver beyond several kings ransoms from south america. The result the wealth made the Spanish decadant and in the process nobody of importance had to strive to make a living and the wealth not only got squandered to the likes of the Church education effectively stopped for several generations. Thus so did real inovation of substance.

After WWII Europe was a basket case the US initially tried to inflict further economic punishment on. Geroge C Marshal became the champuon of a plan to push free trade and 're-industrialisation of Europe. Whilst the bulk of the money went to Britain and France they did not use it sensibly basicaly they spent it on Government policy. Germany however used it to set up a low interest bank that loaned the money out to industry and thus used it as a continuing self funding investment. For other domestic reasons the British Government were seen as wasting the money on vote buying "Beer and Cigarettes for the boys" and the French as wasting it on "The small farmers". For othere reasons seen as those of rivalry and Empire Britain in effect stayed out of the resulting ECC (but remained part of the OECD). The fact that Britain remained at best a stuttering economy outside of the EEC whilst Germany became the dominant industrial heart indicates the failure to invest in industry in a sustainable way and the price that was to be paid for it.

A modern day equivalent would be Venezuela. The oil wealth got spent on social reform without puting inplace a structure that would continue to bring in wealth as the revenues dropped.

In effect the 5eyes have had a glut of data etc, and untill 2013 decadently assumed the wealth of data would continue[2]. Allied with the insistance of smaller government[3] the 5eyes have failed to invest sustainably. The shock of the Snowden Revelations was palpable by all and in effect brought the "going dark" down on their heads. Rather than inovate around the turning off taps they are trying a political FUD campaign via the "Phoney War"[1] on terrorism.

The simple fact is that the FUD is not working to well, and it's been made clear that what ever they propose will be both ineffective in it's stated aim and do more harm than good to the general population.

In short the 5eyes have been off their game for a decade or so as they have not had to try due to the glut[2] and the non IC world has cut their technical lead if not surpassed them in many areas.

[1] See the question I asked the other day about the levels of --supposadly-- thwarted terror attacks in the nations pressing for draconian privacy invation legislation compared to other nations who are not.

[2] Due to the glut of data their technical focus was concentrated on how to process the glut. Or what we would call "big data" something the Five Eye Nations had advantage in, but importantly not constrained to the IC as had crypto and EmSec in times past so they had no advantage. As money talks and BS walks with high fliers the IC had little to nothing to offer the "Whiz Kids", thus industry got the advantage. What the IC ended up with were the pedestrian types that felt they would not make it in academia thus looking for a confortable pension and reasonable standard of living. Solid workers but not trail blazers. Worse is the changing nature of Espionage the success of the Internet had caught most out of the traditional comms view point. The rise in computer comms destroyed most telco models and with it the Secret Squirles on Standards Boards were side lined. Computer hacking and cracking developed their own cultural ethoses both of which clash strongly with "button down" and "BS command hierarchies", respect is earned by deeds not by filling dead mans shoes and schmoozing. Thus those that "can do", "don't fit" in with either "Big-Corps" or their paymasters "Gov-Ents" where kiss-arse creeping&climbing is king.

[3] Another part of the problem is the MIC preasure on politicos to "Cut Big-Gov" so that they get the tax dollars. Part of this are the COTS comparison drives. It's easy to say Gov is wasting money and business knows better. But the reality is Big-Biz does not do Big-Gov systems there never will be COTS secure systems because the market is way way to small to show returns on investment. Thus the MIC traditionaly have $600 hammers and similar to soak up tax dollars as "expenses" not "profit" at the audit interface. They then take the profit at obscene rates one or two steps back from that audit interface.

ab praeceptisAugust 5, 2017 9:55 PM

Clive Robinson, Nick P, RobertT (et al)

Besides some relatively minor issues like e.g. too high complexity "trust modules" etc, I think the problems are mostly of a political nature.

Example: A very core layer of the full stack, namely chips of some complexity, are usually beyond non corp. or state reach. We can't afford to casually have a couple thousand processors made nor can we really test the outcome; it's simply too demanding financially and moreover too much is hidden beyond legal walls ("IP").

Just look at some projects like super-h derivates or opencore. They stumble for years and years before they can have a single ASIC wafer made (and then typically old tech is used) - and in the end they are left to trust the fabs anyway. Or look at Risc-V which looks very promising but the few available processors come from fabless companies or universities and it boils down to the trust problem again.

To make it worse the whole process has given precious little consideration to verification and verifiability, particularly customer or third party. If at all, it almost always boils down to "tested. works. can be sold".

That's (part of) the bad news. The good (not really) news is that the vast majority must hardly be concerned about high grade opponents and attacks. So with some common sense and experience one can go quite far with the fab situation mentioned above. Where with common sense I mean, for instance, to stop building monsters with everything and the kitchen sink on chip but to rather have lots of SerDes lines and non core cpu functionality like network or disk controllers external.

Risc-V looks like a reasonable basis to me. Just not as a soc but as a plain cpu. We must, it seems to me, avoid complexity and rather prefer to have multiple parts with clearly limited functionality. Looking at todays state of technology it seems realistic that that could be done within the board size limits of, say 10 years ago (i.e. it would still fit within ATX format). This would offer the added bonus of being able to have but to not automatically being force fed a whole lot of potentially unneeded functionality (like, say usb ports).

In other words: perhaps I'm just lacking the knowledge wrt. hardware (I quite probably do) but I don't see hardware as the most grave danger (again, for 99.99% of cases). Moreover in my minds eye many of the problems boil down to the same class we know from software. Idiocy and greed mixed with carelessness are example of what I mean.

What really concerns me is this: If we had, just suppose, tomorrow morning the 100% verified correct processor - then what? We'd still have layers upon layers of software crap and hence the end result of crap.

As for Nick P's point, yes, certainly e.g. proven to be reliable and correct compilers are nice. But isn't that somewhat akin to the crypto situation, namely, systems are really breached by cracking the crypto; nope, they are breached by poking ridiculously insecure layers below and above.

I wrote (on another non tech forum and in another context) some days ago that impotence of one party might look like power of another one but is not power. In a similar sense: We (the non spook users) currently are not fu**ed because the nsa is so powerful and smart; nope, it's because we are so impotent - and that is largely due to pretty much the full software stack being crappy.

The most important thing to do seems hence to look at the reasons for that and to get a lot better wrt those. The two major culprits in my minds eye are a) carelessly fumbled and featuritis ridden commercial software and b) (f)oss crap fumbled by well meaning but unprofessionally working people. Which, I admit, pretty much covers 98+% of software.

Enchanting side note: it seems that compilers increasingly do support or work towards supporting Risc-V.

RobertTAugust 5, 2017 10:05 PM

@NickP yeah that's precisely the sort of stuff that I committed to keep to myself, apparently I pissed off a few powerful people and accidentally stumbled onto some ongoing security operations that I should have known better than to dig into.
I only hope I'm smart enough to accept these restrictions and operate within these bounds.

Nick PAugust 5, 2017 11:03 PM

@ RobertT

There still nothing stopping you from commenting on a good chunk of what I posted since it's so public in terms of what I'm saying and others are doing. So, still looking forward to any detailed discussion of any of it.

RobertTAugust 6, 2017 12:14 AM

OK Nick want's proof.
wrt Analog onchip security comprimize methods I'd suggest that nobody is going to try to hide (or for that matter construct) an Analog state machine. I haven't seen a true Analog state machine for over 30 years so it's a fair bet that it's not technology that most hackers would be familiar with.
Where you need to worry is Analog methods to intercept digital signals.
Typically a digital output is connected to another digital input or multiple inputs with various layers of metal routing (interconnect) (modern chips have over 8 layers of metal interconnect.
Now if an analog signal runs parallel to a digital signal you'll often get interference coupled across to the analog signal by capacative coupling, most times this is seen as noise or distortion on the analog signal however if you add a differentiator circuit at the analog input than this circuit can be used to recover the digital signal. If the digital signal in question is something like a carryflag output from the main cpu adder than you are well on your way to constructing a chip level security compromise that nobody will ever find.
The mixing of Analog circuits with digital will ensure that you fall between the cracks when it comes to oversight. Digital guys will swear it's all good and even use words like Proven. Analog guys will just accept that distortion (noise, signal coupling etc) is just a fact of life and probably wonder why you've included a differentiator module but that'd be easily explained away because these guys aren't really ever thinking about security and it would not even occur to most Analog designers that digital interference could ever be used to create a security compromise.

The Analog coupling hack therefore falls neatly through the cracks in the design verification process. The trick is to get the Analog signal route exactly where you want it but since this is typically determined by the Analog engineers or Mixed signal integration team, getting the route where you want it is not that difficult. Assuming the two circuits are in the same physical part of the chip.
If I really wanted to hide this I'd probably "shield" the analog signal by intentionally adding a dummy route. this dummy route signal should be connected to the chip GND but there are reasons for not connecting the dummy route to GND The dummy route itself will typically be excluded from the extraction / verification process because it is not part of the design database, excluded signals are great places to incorporate weaknesses because everyone expects them to be connected to GND but nobody is ever verifying what they connect to. Original database can have the shield signal connected to GND whereas the reworked metal fix can disconnect the gnd connection. and even include a design note to keep this disconnected for Analog reasons.

The chip security team will never suspect a deliberate compromise by an Analog engineer and still be working off the design database that says that signal is connected to GND...oops forgot to update them....see how it really works.


Clive RobinsonAugust 6, 2017 4:18 AM

@ Nick P,

I've found that few EE's seem interested in open hardware. The number of volunteers willing to contribute is always low. It just seems much lower for hardware engineers.

To be honest most of the time making a contribution costs time, rather more so with hardware than it does for software.

If I write a hundred lines of C code I can have a nice Proof of Idea up and running in a day. Further I know that more or less what ever piece of kit you run it on it's going to work for you. Done Dusted Move On (DDMO).

If I design a simple three transistor radio circuit for MW reception I can draw the circuit up on paper in the morning, then raiding my bits box of old tadios and other such parts build it and get it working nicely by the end of the day.

However if I send you the circuit the chances are that you will have no chance of building it or getting it working. The first reason is you will not have access to the Components I used. For instance you might use the right value of capacitor but is it the right type size and leg out? Will you put it in the right orientation in a "Dead Bug" / "Manhattan Skyline" layout?

This means that you need rather more than the circuit you need not just a shoping list but a qualified parts list for a supplier you've got access to. You also need photographs and hand drawings. Even then your chance is minimal to get it working, you need a set of setup notes, you will need similar test kit and even the right kind of screw driver to adjust the variable capacitors (trimmers) and cores in inductors.

I can reduce the layout issues at low frequency by producing a PCB layout. But as frequency goes up you need to ensure you have the right type of PCB material FR4 does not cut the mustard above 100MHz unless you are very carefull in your design.

By the time I've done all of that you are looking at five or six days of work. But it still may not work for you if I've not actually built the circuit a number of times with slightly varying values of component to test that there are not issues with tolerances.

When it comes to transistors and similar there are likewise issues I've got VFETs in my box of bits from IRF they are realy quite inexpensive parts. The older ones will hapilly work in circuits to over 120MHz however those made a few years later will be lucky to work at 40MHz and later still devices will work at more than 150MHz but have stability issues. If you use the IRF test setup they will all pass to the data sheet specs. The issues I see are due to quite minor changes in IRFs production techniques... This means I have to check date codes etc.

All of this takes one heck of a lot of effort for maybe nobody. Yup there's a good chance that nobody will build it because of the amount of time they will have to put in.

I know people who have designed some quite nifty bits of kit, magazines have published it and the designer has got PCBs made and sourced kits of parts and waited for a phone call or letter ordering a kit and given up. Then will have moved house etc to get a letter from somebody forwarded to them ten years down the road asking if they can buy not a kit of parts, just the PCB... Worse a letter saying they bought the kit but only now have got around to building it and have broken the leg of a trimmer or some such.

There is a company called Toko that used to make all sorts of wound components and would sell them in small numbers. They've stopped making a lot of them, and other companies make "look alikes" that are not electrically the same because they have used a different form of support plastic or a different type of ferrite material for the tuning slug, thus the maximum Q you can get in the circuit is insufficient for filters to work properly.

These are just some of the reasons EEs don't do projects for people. Not being funny though would you expext a Renault, or Ferrari race team engineer to write an article on how to add a turbo charger for a five door family hatch back?

But there are other reasons, a lot of FOSS is written as ego food by young code cutters with time on their hands who have been told that a FOSS project on their C.V. will make them of more interest to employers. It might help get a first job or the second but after that it's a liability. The quality of the software by and large will be crap, it will suffer from "Library Hell" and because they want to look smart will use all sorts of "cleverness" in tools and the like. It's not just the coder who will have DDMOed and orphaned the project, the library writers will have DDMOed or made major changes, similarly with the tool chain. That graphics library everybody was saying good things about, now nobody has heard of ir and the Wiki page has spider webs of broken links.

Most EEs who produce open designs are of a diferent ethos. They are "shed men" that is they are married with 2.4 grown up children and a flea bag mongrel, and rather than go down the pub to watch sports they go down the garden to their shed which is also their work shop, where they "keep their hand in". They have moved upwards in EE and if not retired are more senior engineer or managment. They design things for other hobyists because they still remember "how to use the good stuff" and get the most from it. Young EEs are to busy trying to keep a job they don't use soldering irons or touch components. The components they design with come with manufactures recomended layouts and software etc and the minimum order quantity is a tape reel of three thousand or pick and place tray packs of 150 items. To get the components in smaller quantities you have to "Get Samples" and the multiple layers of middle men sales persons are not going to post them to "Fred Bloggs, 23 Railway Cuttings, Cheam" as there are no sales to be made there...

In most cases Open Hardware needs a manufacturing champion who will throw 10-30 grand into a pot to make up kits and throw another 10 grand on getting software and documentation together. It's why the likes of KickStarter exist and as often happens good ideas never show a profit thus die away. The reason they don't show a profit is the young EEs are stary eyed and don't know how to source parts and manufacturing effectively. Such knowledge has a very steep and long curve... Worse they are over ambitious and don't have sufficient indepth knowledge to know what they were trying to bite off.

Have a look at "JackPair" for instance we talked about them here,

https://www.schneier.com/blog/archives/2014/09/jackpair_encryp.html

Three years ago have you seen an actual user review yet?

FigureitoutAugust 6, 2017 10:54 AM

RobertT
--Even w/ infected hardware there's 2 more things you need for a successful implant: 1) Storage of collected information, and/or 2) Transmission of that information to a convienent and covert collection point. #2 is a requirement, #1 for victims that employ shielding and stringent INFOSEC methods and any info from that system is valuable enough to physically go after it w/ black bag attacks. Memory cells may be harder to hide, I'd think #2 is most practical. Otherwise if your hardware backdoor collects and stores info to a tiny buffer/cache of some sort, it'd get overwritten repeatedly and would be almost worthless to an attacker.

Could you also modify hardware peripherals (the transceivers) to essentially create a literal hardware "hidden channel" even if your target was monitoring communications? The most fruitful targets would be ethernet and wifi phy's. My gut tells me no, that that info would be visible but could maybe be hidden in certain parts of a protocol. Or that you'd need a specially designed cable or receiver to receive the info. That would provide a safe exfil channel that could safely operate in the clear.

WaelAugust 6, 2017 11:19 AM

@RobertT,

wrt Analog onchip security comprimize methods...

Welcome back. Sure sounds like the real RobertT. I'm 90% sure it's you. Time will show. I'll be watching you ;)

RobertTAugust 6, 2017 5:33 PM

@Figureitout
Ahmm Yes I know ways to extract the information from within a chip without anyone being any the wiser. That's the beauty of Analog you don't need a huge change in a signal level to communicate information. All you need to be able to do is separate the signal(desired information) from the noise (unexpected unwanted or possibly in this case the actual intended function of the pin).
That's all I'll say on this issue, refer to my post Freedom comes at a price.

As for hardware hidden channels yes there are a few ways to do this which I have discussed in the past. Since none of the laws of Physics have changed, these methods will still work.

@NickP re EE's not contributing to opensource projects. I suspect those that really understand their trade also understand just how pointless such an endeavor actually is. If I know a dozen ways to subvert a design database that will go undetected even with the most rigorous analysis / oversight why would I waste my time, even if I thought it worthwhile how would I convince myself that someone else (similarly skilled) wasn't there with the express purpose of subverting the database. Projects of the type that you're talking about can/do attract the attention of state level players and they're the ones that are most likely to "contribute".

@Wael...I'll be watching you:
I kinda hope you're the only one.

Nick PAugust 7, 2017 11:02 AM

@ Clive Robinson

It makes sense that parts and time being limited might reduce contribution a lot. I do see many active doing projects on forums dedicated to electronics, though. Just not open stuff. The last EE's I met in person had never heard of open hardware. So, it might not have occurred to most of them to even try it due to the culture of commercial engineering where everything is trade secrets and such. Shenzhen is the exception having done FOSS-like stuff with hardware for a long time. I mean, they don't publish but constantly copy. They're cheap, too. So, if it's a funded effort, a reliable team over there being cautious about sourcing components seems like best bet.

@ RobertT

"If I know a dozen ways to subvert a design database that will go undetected even with the most rigorous analysis / oversight why would I waste my time"

It depends on if you want absolute secrecy or security versus relative. I think most hackers not stealing millions from a bank that a nation-state might still be able to attack is highly beneficial improvement over no security or whatever they're using. Likewise for computers on the Internet where the current situation is leading to the largest DDOS's ever seen. Even with restricted networks and leased lines, it would be good to have heavily-analyzed chips designed for improved security. So, it's still worthwhile.

Let's not forget the non-security arguments open hardware that are similar to those for open software. The ability to inspect it for logic-level errors, re-create it if someone stops supplying, improving it for your own purposes, and so on. The market already does this to a degree being competitive on standardized stuff. However, it should cost less if they're differentiating on otherwise open stuff. My concept would be most of the stuff in a SOC that just has to be there would be open. The proprietary would be the integration and differentiating components. Someone might produce open versions of them. These collect on a few process nodes (350/180/90/45/32/28) keeping cost low with maybe eASIC or Triad jumping in to make that last step even cheaper for fast-moving companies.

AndyAugust 8, 2017 12:52 AM

.... And since when is the info in those dumps unimportant? I've seen it send patient data from when I worked at a hospital, I'm sure the gov loves a crash of excel, with a csv and a list of terrorist names.

CassandraAugust 8, 2017 12:57 PM

I am not a hardware expert - if anything, I'm not an expert in anything, so with that caveat aside, I'll make what might be a stupid suggestion:

Would it be (a) easier to validate a FPGA piece of hardware and (b) implement a secure CPU on that? Or is the issue validating MMUs, memory, video controllers, network controllers etc?

Someone has implemented a Zilog Z80 on an FPGA 'from the ground up' - see http://baltazarstudios.com/z80-ground/ and http://baltazarstudios.com/z80-cpu/ - so while a Z80 is not the epitome of computing sophistication, it demonstrates proof-of-concept: virtualize the secure hardware on an easily validated, modular substrate. I realise that there is a colossal performance hit: but if you want to securely encrypt text rather than a 4K video stream, it might be workable.

The above website is part of a home-built CPU web-ring, which makes for an interesting diversion : http://members.iinet.net.au/~daveb/simplex/ringhome.html

As for doing things with/via Shenzen, I'd recommend reading the on-going story of the development of the EOMA68 card computer here: http://rhombus-tech.net/ and the development updates on crowdsupply.com here: https://www.crowdsupply.com/eoma68/micro-desktop/updates - it is not simple, straightforward, or fast.

Building low-volume, custom hardware to exacting standards is hard. It's also worth reading the story of the Pyra - I'm hoping to buy one. https://pyra-handheld.com/boards/pages/pyra/

Nick PAugust 8, 2017 4:01 PM

@ Cassandra

I just posted an analysis on risks, mitigations, and FOSS issues in ASIC's. That's here. The link at the bottom explores considerations in getting FOSS work done.

WaelAugust 8, 2017 4:32 PM

@Nick P,

I just posted an analysis on ...

It says 644 days ago. That's over one and a half years ago!

You posted something similar here, too. The word 'protect' needs to be elaborated on. How do we 'protect'?

Non Sum AggregateAugust 8, 2017 9:40 PM

"-Even w/ infected hardware there's 2 more things you need for a successful implant: 1) Storage of collected information, and/or 2) Transmission of that information to a convienent and covert collection point. #2 is a requirement."

Enter the Intel ME, google, Apple, NDA'd NSL's...

You know the Ken Thompson hack idea is like, what, 50 years old now?

We still have the same gatekeepers with the same vulnerabilities to a powerful state entity, or even a non-authority knowledgeable entity.

The internet is a tool that will at some point be turned into a pure weapon. The point at which this happens will vary from nation to nation, as will the reporting of it. It's just too damn convenient to make any contingency plan that doesn't involve it.

Which effectively makes all of us a captive of it. Whoever owns the ground owns the wires, and whoever owns the wires owns the current, and whoever owns the current owns the data, and whoever owns the data might as well own the world, unless they blow it.

ApokrifAugust 10, 2017 2:13 PM

@Clive Robinson

> all the VMs have clock signitures exactly thr same. The crystal will drift up and down in
> frequency due to things like external temprature and changes in the CPU load. But unlike
> real independent computers all the VMs will remain in perfect lockstep timing wise. Thus
> the simple use of a basic script kiddy enumeration (ping) attack will revel the VMs
> keeping time synchronisum, from ordinary networks with multiple independent machines thus
> crystals thus timing information.

Is it possible to slightly and randomly change each VM's time?

DarrelAugust 12, 2017 11:35 AM

I find this collection of error messages fascinating, but at least within the dynamics of the United States, I think its far more likely that Microsoft simply provides that information when asked.

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of IBM Resilient.