Friday Squid Blogging: How Scientists Captured the Giant Squid Video

In June, I blogged about a video of a live juvenile giant squid. Here's how that video was captured.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Read my blog posting guidelines here.

Posted on September 13, 2019 at 4:05 PM • 38 Comments

Comments

SpaceLifeFormSeptember 13, 2019 5:12 PM

SimJacker

https[:]//thehackernews.com/2019/09/simjacker-mobile-hacking.html

Note that many US cellcos are twisting or obfuscating on this already.

Ismar September 13, 2019 5:51 PM

@Anders
Thanks for sharing .
Very interesting read about yet another side-channel attack due to optimisation of the CPU workings. Surely, it is be something nation state actors would be actively involved in exploiting.
Now that we know how this works one easy way of mitigation is to type your SSH passwords having a fixed wait intervals between each letter but how practical would that be as long term solution 😀?
BTW - this has nothing to do with using of the popular network tool netcat

Alyer Babtu September 13, 2019 7:57 PM

@Anders

Re R. Stallman

He seems to miss the point that “underage” is not just a number but rather not able to meaningfully be willing or unwilling because lacking the required maturity.

Clive RobinsonSeptember 14, 2019 5:50 AM

@ Anders, All,

From the ZDNet article,

    INTEL DDIO FEATURE AT FAULT At fault is the Intel DDIO feature. This is a CPU speed optimization feature that was specifically designed for Intel's line of server-grade processors.

Yet another little hardware SNAFU from Intel's Marketing Dept "Go Faster Stripes" policy, that also gave us the "Xmass Gift That Keeps Giving".

But it's even dumber than usuall,

    DDIO works by allowing peripherals, like the network card, direct access to write data inside the CPU cache, instead of RAM, as peripherals normally do.

That "normally do" sounds almost benign, but it hides a long history of glaring security mistakes made over and over again. Because this type of security fault lies below not just the CPU level in the computing stack but in effect below all the Core Memory (RAM) hardware protection mechanisms. Thus alows "bubbling up attacks" that can not be stopped by all those formal methods you hear talked about. That is there is no top down security mechanism you can use. You have to have a low level hardware protection mechanism.

A little history for those under twenty in Y2K, back several decades ago when 8bit computers were still common, some one came up with the idea of Dynamic Memory Access (DMA) to move data in memory when the CPU busses were in inactive cycles (ie before "pipelining" had been brought to CPU chips). Even back then it was known that DMA was a serious security risk if it became available outside of the system security perimiter. But it gave a big performance boost when doing internal things like Graphics, so was considered an acceptable risk. Because after all who would need that sort of performance outside of the system box anyway...

A few years later a little over a third of a century ago Apple with their Firewire serial protocol (later IEEE 1394) ended up using DMA to get the then high speed data into memory via DMA and in their "need for speed" designed it to entirely bypass the OS and higher levels of hardware in the computing stack thus all system security... Unsurprisingly it got hacked several times and became famous for it, even though the same problem existed for nearly all extetnal hardware ports at the turn of the century.

It became such an issue that low level IOMMU hardware was developed in the CPU silicon to try to limit external hardware getting access to sensitive "Core Memory" (RAM) where all sorts of kernel security mechanisms such as Memory Managment Page Tables, crypto keys and worse are stored.

Thus nobody can say that DMA Attacks are new or unknown, nor their severity of "total system access" equivalent to "Ring -3"... The only difference here by the looks of it, is that Intel appear to be going for even more speed than DMA gives by talking to "Core Memory" or RAM by using DDIO to talk instead directly to cache memory inside the CPU silicon, just to make those "Go Faster Stripes" even wider and a lot less secure than before[1]

Like DMA attacks have been long known, Cache Timing Attacks are likewise long known and predate even the AES competition[2] and as such are one of the ways to get either plaintext or keytext around a security end point. Thus anything that effects Cache Memorry timing is security wise a very large "NO NO". Something that most computer security people involved with design should be aware of.

As for the use of "interactive" Human Computer Interfaces (HCI) it's been long known they are a double threat to security. It's not just the cadence issue of peoples typing it's also an issue of small alphabet size turning otherwise secure encryption systems into the equivalent of a simple substitution cipher unless certain types of encryption mode and plaintext expansion techniques/protocols are used. Whilst many systems these days take precautions against alphabet size issues, timing side channels that haemorrhage cadence information are not. It's something that has been discussed on this and other blogs in the past, and just as with AES time based side channel issues it's one of those things that repeatedly "falls on deaf ears". Which when allied with "fall back attacks" of legacy systems[3] makes systems vulnerable for a quater century or so...

As for the Intel comment of,

    "Additional mitigations include the use of software modules resistant to timing attacks, using constant-time style code,"

Either the person has no clue as to what they are talking about or it's a thinly disguised "Two Finger Salute".

Thus the comment by VUSec researchers saying that they,

    dispute that using side channel-resistant (constant-time) software would help. Instead, they recommend disabling at least RDMA on affected CPUs, since this reduces the attack's efficiency.

Which again highlightes Intel's increasingly obvious lack of respect for their customers, their security, and much more.

My advice would be do not use either RDMA or DDIO unless you have a very real use case[4] THAT CAN also be protected by other security mechanisms such as reliable segregation. As the latter can not be achived by "On-Line", "Cloud", or most "Hosted" service suppliers this type of attack remains a risk with the only solutions being "In-House Resources" where clear and enforcable security perimiters can be established.

[1] In essence what Intel have done with DDIO is not just bypass the RAM and OS, but also more than a third of a century of security knowledge that says it's such a bad idea even Intel with all their other low level hardware SNAFU's baked protection into their CPU silicon with IOMMU...

[2] Whilst known by a few people prior to the AES competition it was not widely talked about outside of certain circles (as is often the case with side channel attacks). What was clear to some however was that the NSA through NIST fixed the AES contest to make not just time based side channels much more certain but in ways that would almost transparently propogate through a computer system and become quite visable on IO such as networking. Worse by the rules of the competition it virtually guarenteed that insecure versions of AES would be built into virtually all code libraries and thus consumer products.

[3] One of the lesser joys of security design is the "It must be compatible" edict when also tied to the "Don't confuse/scare the user" edict giving rise to "silent protocol fallback attacks". What happens is a security issue is found at protocol revision N thus it and all earlier versions now have a known security vulnerability. Which gets fixed in protocol revision N+1 or later. The problem is the installed user base with revision N or earlier can be very large, and might well be in many products that can not or will not be patched or upgraded in the next quater century. Such as the likes of Industrial Control Systems (ICS) and things like consumer meters in utility "Smart Grids". This means that your software product has to work with revisions before and after revision N. Thus an otherwise secure software product will "fallback" to the insecure protocol and to avoid confusing users the software products will not give the user any real warning things are not secure (warnings like open padlock icons on different colour backgrounds is a recipe for insecurity as are most status messages). The thing about all fallback mechanisms is that they are easily invoked by a third party that can do a Man In The Middle (MITM) attack which can with remote systems often be trivially accomplished... I've actually seen more than one software product "fallback" to "plaintext" communications with a simple MITM attack on a firewall at the security perimiter gateway, thus would also work at the upstream router or any other node between the user and the system they are accessing...

[4] Is there a real need for such "Marketing Specmanship" hardware such as DDIO? In most use cases absolutely not, which brings up the question of which use cases predicate the likes of DDIO. Well... It's large computing clusters with need to move bulk data as a premium[5]. This is not a large number of people and falls into the likes of High Performance Computing (HPC) and On-Line / Cloud / Hosted service providers. Who by their very nature can not use the security procedures required to stop time based side channel attacks on the likes of user typing cadence due to it's very long time intervals.

[5] Moving data has always been a bottle neck it's why CPU's have "registers" and the likes of early HPC "Super Computers" had "register files" that alowed vector processing. In later systems the various layers of "cache memory" helped increase data through put. There are two fundemental issues with moving data "Distance" and "Bandwidth" you try to minimize the former and maximize the latter. Appart from moving data storage directly into the ALU there is very little you can do about distance these days. Which means due to the speed of light there are very hard restrictions on how fast you can move data over distance. That is there will always be "data movment latency" from a request to delivery. The only way you can lessen distance effects is to bring as much data at the same time as possible, in the past we used to see this done by increasing "the data bus width" or by having "multiple concurant data paths/channels" but even this has hard physical limitations it's why some HPC people talk about "Photonic Computing" built around "photonic integrated circuits" and "photonic/optical multidimensional waveguides"...

VinnyGSeptember 14, 2019 7:06 AM

For any readers who have continued to use Win 7 or 8.1 in lieu of moving to a *nix OS: According to Windows savant Woody Leonhard, Microsoft has been bundling telemetry code in recent Windows "security only" updates. The code was included in the July updates, absent in August, but resumed in September. Leonhard, who is largely a Windows proponent, claims that MS' stated justification for including the code is quite weak. Is it spyware intended to give MS information that it intends to use to try to cudgel-update 7 & 8.1 users to Windows 10 (yet again)? Of course, Windows 10 has had its own update drama of late: an update affecting Cortana and Search that caused those services to spike CPU utilization has been "fixed" by an update that eliminated the performance issue by breaking Search :)
https://www.computerworld.com/blog/woody-on-windows/

AlenjandroSeptember 14, 2019 9:28 AM

Somewhat interesting article from a business website helping us to understand why facial ID is the inevitable future and why "they" (coporations and governments, everywhere) must have it on THEIR servers, and not on the user device:

Sounding The Password’s Biometric Death Knell

"Pointner cautioned that whatever replaces the password must be recoverable. Nowadays, he said, it’s relatively easy to recover and replace passwords, and yet that’s not the case with biometrics in the case devices are lost, stolen or destroyed.

One way to deal with that is to store the biometric data on the cloud and on servers, rather than on the device itself, so it can automatically be backed up, and where technical malfunctions need not hinder consumer/business interactions.

“The data that is stored should be obfuscated in a way where you can never present it back to the system so that the system would say, ‘Oh it’s you again. Hey, I’ll let you into this account.’ That’s a critical step in terms of security,” he said, thus preventing what are known as playback attacks."


OK, there's a concession the face ID data and key must be "obfuscated" on the cloud server. Frankly, I don't think that's possible. But, what is possible is a pretend way to do that with a standard lawyer-weasel-word-TOS, like "we promise to be very careful".

I have no trust whatsoever in biometric authentication or the cloud. Yet, that's the future.

My question would be, how do you resolve the trust issue?

AlejandroSeptember 14, 2019 10:18 AM

Speaking of passwords, I realize this discussion is very old but,

Right now I use easy passwords for low sensitivity logins, like to the one time website to get the free copy of their newsletter. For the important stuff ($$$), I do what I can to make it very hard.

When Face ID becomes the universal identifier, what happens when "they" lose it. And of course when it does get lost it will be to some off the way website, in the cloud, over some low priority login managed by a third tier contract employee in the third world.

I am not going to beat this too far, but a question becomes, What happens when "they" get your face, how do you "change it"?

VinnyGSeptember 14, 2019 10:26 AM

@ Alejandro re: "What happens when they get your face...?" As if cosmetic surgeons aren't already wealthy enough.s I guess you could get the local organized crime guys to give you a "makeover"...

Sherman JaySeptember 14, 2019 3:28 PM

Facial recognition is a very dangerous issue for many reasons as pointed out by the contributors to this blog. Companies and governments abuse it. The 'artificial intelligence' face, mood, etc. recognition is a vulnerability. In a recent test facial recognition matching MIS-identified many congress people as criminals.

So, obviously, we should trust our facial ID to be 'safely' backed up by a third party in the cloud! /sarcasm much.

Could we hold up a picture to the camera of some (not so famous) person out of history as our "ID"? That just might thwart a lot of grief if it works. (Maybe I'll use a picture of Mussolini) Maybe even a picture of a face out of an old artwork that we feel would be hard to guess might help.

Alright, those of you that post here all have keen minds. Let's hear what you think about that idea.

>> Also, there is the BIG problem that most of us in the united states that have driver's licenses have already had facial recognition profiles built up by the government, google, facebook etc. All the state governments have already allowed those entities to get copies of our driver license info and photos.

AlejandroSeptember 15, 2019 5:48 AM

@Clive Robinson

That was absolutely marvelous. What can the citizen/slave class do to resist corporate nation states? Is defeat inevitable? (briefly)

VinnyGSeptember 15, 2019 7:33 AM

I recently saw a request on a different blog I frequent from someone whose email address and password was involved in a large internet breach for recommendations to mitigate the potential effects. He asked about Lifelock service, which I recommended against, as I have seen it widely panned. I made a few elementary suggestions about collecting accounts, passwords, email addresses, etc that were "tangential" to the breached account and taking whatever account modification or deletion steps that appeared prudent as a result. In the process, it occurred to me that it might be generally useful to have a personal database application that correlated not only account passwords and sites, but categorized in some summary form the required personal information collected there. Anyone know of such an application? Is this a valid concept? Would it be worthwhile to expand an application like PasswordSafe to add this information/function?

vasp pupSeptember 15, 2019 12:55 PM

@Clive said: "But ask yourself this, how difficult would it be to manufacture an automated submarine?"

The answer is in the video below:
https://www.youtube.com/watch?v=dOB_IdTeen4

@all responding to my initial post on drones attack in SA - Thank you!

As new technology make possible of weaponizing commercial drones evolved, it requires reconsidering all paradigm of protection critical infrastructure (data centers, water supply factories, electric power stations, manufacturing facilities like in the last attack, key government buildings - you name it)in countries involved in conflicts or already targets by international terrorist groups) should add physical protection against drones attack including monitoring, jamming, destroying, etc. as UK did for airport recently and US for nuclear plants and military bases.

Sherman JaySeptember 15, 2019 1:30 PM

@Clive (as his usual brilliant self) brought out a couple of important and more basic additional points to the eternal conflict that is the feudal system that still persists everywhere. We are all prisoners of war, whether it is a water war or an armed conflict or the economic warfare waged by the wealthy/powerful against the populace.

The big question that I constantly work to answer (with out much success so far) is, as @Alejandro pointed out, how can we end that model of aggression and abuse that runs/controls human society? The drive to perpetuate it, participate in it and benefit from it at the cost of the populace seems to be an ingrained primary drive in many selfish, soulless minds.

(I'm going to carefully climb down off my soapbox for now) But, in my own limited way, I will continue to work to create a more equitable and enlightened world. I thank all those who have that same vision.

FUN
https://www.gocomics.com/shoe/2019/09/15

and

GAMES (not the good kind, the silencing of whistleblowers to cover-up crimes leaves us all less secure)
https://readersupportednews.org/news-section2/318-66/58703-rep-adam-schiff-subpoenas-acting-intelligence-director-over-whistleblower-complaint

Sancho_PSeptember 15, 2019 5:30 PM

@Sherman Jay
Re:
”… how can we end that model of aggression and abuse that runs/controls human society? The drive to perpetuate it, participate in it and benefit from it at the cost of the populace seems to be an ingrained primary drive in many selfish, soulless minds.”

We can’t, because the world can not support our western lifestyle for all humans already living on this canoe / island. He Wa’a He Moku, …
The “model of aggression and abuse” (= ruthlessness, part of our firmware) prepares us mentally for the upcoming unavoidable reduction of “others”, be it by nature or otherwise.
The gloves are coming off.
(NaZZionalism stems from the same issue: We must be first.)

FaustusSeptember 15, 2019 6:29 PM

@Alenjandro

I don't understand how you can lose your biometric signature. Can't you just measure yourself again? Isn't that how your identity is verified?

I don't see any inherent problem with passwords used well. All the replacement and supplemental systems in use seem to also be subject to breaches.

Privacy is down the toilet if you tie all your accounts to immutable personal attributes. Biometrics will always be subject to increasingly sophisticated replay and spoofing attacks. And you can't revoke your credentials when compromised.

I really think this anti password propaganda is largely propagated by people selling other identity systems. Can somebody point to an independent study that quantifies the security of various account access authentication options?

Maxwell's DaemonSeptember 15, 2019 6:44 PM

@VinnyG:

I have used PasswordSafe (PC and Android versions), which Bruce had a hand in developing. since forever. It has a Notes field which is where I keep ancillary information.

SpaceLifeFormSeptember 15, 2019 6:46 PM

Is broadcast radio infrastructure backdoored?

Just observed an incident where website info outraced broadcast radio.

By over 10 seconds at least.

Info was same, and accurate.

Does radio have same mandatory delay like tv?

If so, then why can a website be exempt?


Clive RobinsonSeptember 16, 2019 2:14 AM

@ Bruce, and the usual suspects,

The "Low Hanging Fruit" appears to have changed in Ransomware attacks accordong to F-Secure,

https://www.zdnet.com/article/ransomware-attacks-weak-passwords-are-now-your-biggest-risk/

In an investigation earlier this year they found that fishing attacks are now not as popular as weak password techniques.

Why this is the case is not investigated in the ZDNet article, however it is important information if it can be deduced.

My personal view is it probably is related to the increasing number of entrants into the Ransomware area of criminality due to the fact that "pay-or-die" is realy the only option for most organisations that get hit.

The simple fact is that Ransomware attacks work because for what ever reason organisations do not take steps to mitigate it and attack are a quite real existential threat to way to many entities.

Whilst the usual industry argument is "AV and patching" people should realise that neither of those work against Zero-Day based attacks. As we saw with WannaCry there are often reasons why patching can not happen and the history of AV software is that covarage of known malware is often not compleate by any given vendor.

Thus it's reasonable to assume that Ransomware will get past the "AV and patching" defences and will only get worse if Governments get their "backdoors" (some already think that some Governments have applied coercion to AV companies to do this already).

There are however mitigations that are known to work or atleast minimize damage. So if people design their systems with these mitigations in mind they will have greater resilience.

ATNSeptember 16, 2019 3:42 AM

3D Secure 2: Presented in French radio as "use your PC to authenticate Internet payments", obviously your PC is secure...

https://stripe.com/gb/guides/3d-secure-2
The enforcement of Strong Customer Authentication (SCA) in September 2019 makes 3D Secure 2 all the more important if you are doing business in Europe. As this new regulation will require you to apply more authentication on European payments, the improved user experience of 3D Secure 2 can help reduce the negative impact on conversion.

NiclSeptember 16, 2019 4:32 AM

@Clive Robinson wrote, "Further that inflation that devalues money is quite deliberatly designed to seperate further the gap between rich and poor via "fixed wage labour -v- rising consumable/asset prices". Thus those who minimise their consumable expences in favour of aquiring assets end up over time getting wealthier. "

This could be true if there weren't a devaluation cycle that go along with it. Inflation is a stretched out end result of perpetual credit bubble cycles. Thus, the inflation of consumable/asset prices is merely a side product of this phenomenon. In a fiat money system, inflation means more debt because money stems from creation of various debt instruments backed by worthy credit. A race to the bottom of asset prices occurs when bubbles burst which results in confiscation of worthy assets with the bad ones written off the books or put on the public's balance sheet.

Ergo SumSeptember 16, 2019 5:27 AM

@Faustus...

I don't understand how you can lose your biometric signature. Can't you just measure yourself again? Isn't that how your identity is verified?

The authentication method does not matter much, as long as the client devices and authentication servers security are not addressed. Biometric data can be compromised, just like passwords. Client devices get compromised and data breach for authentication server are pretty much a daily event nowadays. Does it really matter, if the compromised system exposing passwords or biometric data?

The issue with biometrics, that they can't be altered. If records of your fingerprint or face, or iris, are compromised, you can't realistically reset your face or your fingerprints. Hackers can remotely steal fingerprint/faceID, or athorities forcibly make someone unlock their device/account with fingerprint/faceID.

From the 2015 Black Hat:

h**ps://www.blackhat.com/docs/us-15/materials/us-15-Zhang-Fingerprints-On-Mobile-Devices-Abusing-And-Leaking-wp.pdf

Password had been with us for a very long time and in my view, it's going to be with us for a long time for simple reason. No other authentication methods provide as easy replacement for the compromised credentials at as cost effective way as replacing the password.


Ergo SumSeptember 16, 2019 6:09 AM

@WinnyG...

For any readers who have continued to use Win 7 or 8.1 in lieu of moving to a *nix OS: According to Windows savant Woody Leonhard, Microsoft has been bundling telemetry code in recent Windows "security only" updates.

Microsoft has been doing that since 2015 for W7 and 8.x. The built-in spyware (telemetry in MS definition) actually started in W8.1 and perfected in W10, where it cannot really be disabled.

Or more accurately, disabling the spyware is a monthly event. All updates reset the disabled telemetry settings to enabled. That's quite evident by looking at websites that have detailed instruction for disabling the telemetry, or the apps that do the same through a GUI, like ShutUpWindows10. All of them recommend going through the process of disabling the telemetry functions after any MS updates. It's sort of useless to do this, since by the time the end user disabled the telemetry futures, all the freshly created telemetry data has been uploaded to Microsoft servers. MS would prefer real-time spying, and does that with the ignorant masses, but settles for monthly updates for the tech savvy people.

The link referenced "bundling telemetry code" is just code upgrade with additional "features", a.k.a. as more extensive spying for w7 and 8.x.

Microsoft has a good business reason for doing this. The data collected is much more extensive, as such more valuable, than the browsers collected data on people. In my view, this is the main reason why MSFT stock increased from the perennial $25 to the current $136 per stock.

VinnyGSeptember 16, 2019 9:18 AM

@ Maxweil's Daemon - re: PWS Notes field - Thanks. I am well aware of the Notes field, I was wondering whether something a bit more structured might be useful; possibly some small footprint, lightweight SQL table that could be linked to fdrom within PWS.

TatütataSeptember 16, 2019 9:29 AM

Zach Dorfman, Jenna McLaughlin and Sean D. Naylor : Exclusive: Russia Carried Out A ‘Stunning’ Breach Of FBI Communications System, Escalating The Spy Game On U.S. Soil, Huffington Post, 16 September 2019

It was around this time that Putin’s spies in the United States, operating under diplomatic cover, achieved what a former senior intelligence official called a “stunning” technical breakthrough, demonstrating their relentless focus on the country they’ve long considered their primary adversary.


That effort compromised the encrypted radio systems used by the FBI’s mobile surveillance teams, which track the movements of Russian spies on American soil, according to more than half a dozen former senior intelligence and national security officials. Around the same time, Russian spies also compromised the FBI teams’ backup communications systems — cellphones outfitted with “push-to-talk” walkie-talkie capabilities. “This was something we took extremely seriously,” said a former senior counterintelligence official.


The Russian operation went beyond tracking the communications devices used by FBI surveillance teams, according to four former senior officials. Working out of secret “listening posts” housed in Russian diplomatic and other government-controlled facilities, the Russians were able to intercept, record and eventually crack the codes to FBI radio communications.

That seems to be a big story. I find it curious that it was published in the HuffPost as an "exclusive", considering that the authors have a number of affiliations, and presumably could have proposed it to more prestigious media. In any case, large print and TV outlets don't seem to have picked it up yet.

The use of the conditional in the above text sounds like this would have been more of a SIGINT rather than a COMINT one, and I can't find a passage stating that FBI networks were actively penetrated.

These news strangely sound like a plot line from The Americans, S01E05 "COMINT (2013)", in which the deep cover agents try to obtain intel on a new FBI radio.

This story would be the real reason for the sudden expulsion of Russian diplomatic staff in the waning weeks of the previous administration. If this were really the case, and the reproached activities were limited to interception from consular compounds, then it would be another case of pots calling kettles black, as US foreign missions have a long history of suspected shenanigans...

JonKnowsNothingSeptember 16, 2019 10:27 AM

@Clive Robinson et Co

for what ever reason organizations do not take steps to mitigate it ...

Recently I was discussing similar issues that occur in Construction Industry as well as Computer Systems: SHODDY practices.

Construction shoddy practices are legion.

We have accepted SHODDY Systems as normal:

  • Computers don't work.
  • Software doesn't work.
  • Security doesn't work.
  • Applications don't work.

The list of what doesn't work is long. The list of what does work is so small it might not be a pile big enough to find.

I observed a person using a professional high grade software system that had some good restrictions set up for data level access. A low level access person was not able to delete an item. The message box popped up saying "get authorization". The person mindlessly clicked the delete option 10x more assuming that some how the computer would "fix itself".

The reason is: Shoddy software is built on "try it again and see if it works THIS time".

People have been trained to accept shoddy goods and services and to accept it as part of the price tag.

Want that kewl looking new phone? Pay $$$$. Ahhhh it broke, too bad. Pay me again.

Sort of like the Ransomware where you don't have any choice but to pay and pay and pay even though it still won't work.

Clive RobinsonSeptember 17, 2019 3:28 AM

More troubles in Linux Random

In a saga that appears to never end Linux has problems with it's source of randomness and the way people (ab)use it. So there is the following that has popped up,

https://lore.kernel.org/linux-ext4/20190915081747.GA1058@darwi-home-pc/

For those not close to or in the loop on "random in *nix" distributions, get a hot cup of your favourite "Brownian Motion Generator" sit back relax and read on,

Historically like all Commercial OS's from the miriad of platform vendors of the time that had a random generator built in, *nix /dev/random was a very fast but very insecure algorithm often a simple "add C, mod N" linear generator (see Vol 2 of Knuth for a discussion of the failings).

Sort of great if you are doing simulations and needed to do things the same way twice or more, but usless for doing anything even remotely secure where "The difference between determanistic and nondetermanistic behavior in the observed output is very important"[3].

The random problems became so obvious, that at some point all *nix needed to change but by then the various commercial *nix were in the thick of fighting pyric wars etc. So like a stop motion film of a fragmentation grenade nearly all *nix went their own way doing considerable damage in the process... Which in turn started a history of hack upon hack as people realised it was not an easy problem with so many corner cases that you had to assume nearly every use case was a corner case unless you could definitely show otherwise. So a real problem even if you were developing quite simple code for multiple platforms.

To this day there is still no commonality of function across the various *nix platforms and historically the Linux one that appeared worst was the Debian GNU/Linux as that's where a lot of the problems --that other *nix had as well-- first showed up.

Well it appears that Linux random(4) is still unsuprisingly having teething problems after several decades...

An ever changing part of the problem is "Where do you get entropy from?", and that alone is a subject of great philisophical as well as technical debate. Especially as technology keeps changing (think mechanical HD's to Solid State HD's as just one of a myriad of such changes where nondeterminism from physical sources effectively disapears).

In Linux, there was --prior to 4.8 when a CS-PRNG was added to the mix-- a random entropy pool and the only difference between /dev/urandom and /dev/random was what happens when the base random process thought there was insufficient entropy in the system /dev/random fails closed and returns with a value, /dev/urandom fails open and blocks waiting on the entropy count unless you made inadvisable changes via the IO Control (ioctl) mechanism. Basically both drivers were getting their mechanical and other nondetermanistic entropy from add_disk_randomness(), add_interrupt_randomness(), and add_input_randomness()[0].

Other *nix's actually fudge around this lack of entropy during re-boot by reading in entropy stored in a file (you can in theory do this on Linux via the ioctl call but I don't know of anyone who has tried it). However this does not work on "First power up" on embedded and similar systems such as network appliances as the "file" either would not exist or have "Content known by others". Such as the manufacturer or worse the likes of the NSA, GCHQ and any other entity that decides to take an intetest.

Another part of the problem is "documentation" it's often out of date or plain wrong. This has given rise to a whole myth based view not just of "Here be Dragons" but how you should use the Linux random system. So much so it has it's own documentation[1] that people should read.

At one point the correct advice given in the random(4) man page[2] was,

    When read during early boot time, /dev/urandom may return data prior to the entropy pool being initialized. If this is of concern in your application, use getrandom(2) or /dev/random instead.
    The /dev/random device is a legacy interface which dates back to a time where the cryptographic primitives used in the implementation of /dev/urandom were not widely trusted. It will return random bytes only within the estimated number of bits of fresh noise in the entropy pool, blocking if necessary. /dev/random is suitable for applications that need high quality randomness, and can afford indeterminate delays.

Is not exactly clear, but yes /dev/random is both depreciated and required for certain security related tasks...

Often Crypto tasks such as making key certificates and starting up certain crypto processes all of which if done at boot time or early in the powered up state will not work correctly. Because the entropy estimator has blocked /dev/random and developers do the wrong thing such as use /dev/urandom or getrandom(2) to keep users happy...

This is actually a realy bad security issue in the likes of "embbeded systems" or "network appliances" especially at "first power up out of the box". Which is a time when all the PKcerts and other Master Secrets are usually generated... Something I'm sure the NSA, GCHQ et al must love.

Well things have not been good in that respect for some time. Nearly all mechanical sources of entropy are virtually gone in many systems, as is user input, the harware RNG's in CPU's are either broken or untrusted and due to fundemental choices in design Linux's RNG blocks during the boot process often waiting what feels like eons on entropy which is fatal on "Network Appliances" and the like.

So the Linux random system is far from what it could be which is why it's on the move again, if they get it right or not is however "A toss of a coin, in a moment of time".

[0] For information on the actual sources of mechanical and nondetermanistic input to the main entropy pool on your distribution and revision of GNU/Linux, you will need to take a look at /drivers/char/random.c in your kernel source tree.

[1] https://www.2uo.de/myths-about-urandom/

[2] http://man7.org/linux/man-pages/man4/random.4.html

[3] Why is a nondetermanistic output so crucial, well because you often use it for creating PKcerts and Master Secrets/Keys on which your security rests. You can read further on the subject from our host @Bruce, who's docs on Fortuna and Yarrow can be found at,

https://www.schneier.com/academic/fortuna/

Also further reading is to be found in "Mind your P's and Q's"

https://www.usenix.org/system/files/conference/usenixsecurity12/sec12-final228.pdf

Sherman JaySeptember 17, 2019 2:10 PM

@Sancho_P,
I want to believe that, and am looking for, ways and places where we can exist and contribute that are governed by the best, not the worst, aspects of human characteristics. I believe that many of us wish to aspire to more honest and caring ideals in our lives. Many of those that post here are doing what they can to help us create a world of greater security and privacy.

I like the responsible tone of the site you linked to.

However, as many here point out, the greed of corporations and governments work in a manner contrary to a world where people are safe and secure.

Sherman JaySeptember 17, 2019 2:27 PM

@Tatütata,
and all the other participants here, is there more information on the 'stingray' devices recently found all around washington D.C? How long were they in operation and who controlled them? That matter is another concern of compromise of government (and private) communications.

I see the phrase 'big brother is watching you watch' and the debacle of police departments and the profusion of Amazin RING doorbell cameras (to what extent are they security and spyware?) and wonder how deeply our world is slipping into everyone spying on everyone else.

vs ppSeptember 17, 2019 2:35 PM

Gatwick Airport commits to facial recognition tech at boarding
https://www.bbc.com/news/technology-49728301

A spokeswoman for Gatwick said it had designed its use of the technology to be "compliant with all data protection law" and passengers would be able to choose to have their passports checked by human staff.

vs ppSeptember 17, 2019 2:59 PM

Edward Snowden: Germany a 'primary example' of NSA surveillance cooperation

https://www.dw.com/en/edward-snowden-germany-a-primary-example-of-nsa-surveillance-cooperation/a-50452863

Edward Snowden was in his mid-20s when he joined the US National Security Agency (NSA) in 2009.

"In his new book Permanent Record, he describes working at "America's premier signals agency" as being "a dream job." He also writes of how he uncovered STELLARWIND, which he calls "the deepest secret of the NSA."

The program was launched after the September 11 terrorist attacks. Snowden claims that, contrary to what the authorities say, it never ended. Instead, it became an instrument of mass surveillance and went "from using technology to defend America to using technology to control it by redefining its citizens."

Sherman JaySeptember 17, 2019 5:27 PM

@C U Anon,
Interesting info. Bruce's blog is much more open (less spying than most sites) and allows most honest comment (and that privilage is seldom abused) compared to the limitations you pointed out on the other site. Thanks for looking into that other site for us. I certainly don't consider Bruce's blog an 'echo chamber' since there is such a wide variety of perspectives voiced here.


@Anders and @vs pp,
Also, I must state that I think (contrary to the gov't) that honest whistleblowers are a critical element to trying to reduce corruption in institutions. Their persecution tells me that they are exposing something the powers-that-be want to keep hidden from the populace. Keeping it hidden might be a legitimate 'national security' issue in some cases, but often it is likely to hide improper activities. How to determine which it is always takes some careful objective research.

JonKnowsNothingSeptember 18, 2019 1:50 AM

@Sherman Jay

is there more information on the 'stingray' devices recently found all around washington D.C?

Here are some generic YMMV "answers":

  • stringray and dirtbox (DRT) devices are now a dime a dozen.
  • any group or any affiliated group to law enforcement likely has them.
  • loads of non-law enforcement folks have them too.
  • cell tower sim devices of various types are harvesting your cell data in nearly every location you go. These may be "lessor" devices but they still get you where they can track you.
  • there are so many surveillance systems, devices, cameras that no one really knows who owns them. They don't know who put them on the towers. They have no idea.
  • it is not uncommon practice for a Pen Trace to be "accidently on purpose" forgotten about even though the "authorization" has expired, the device is never removed and is in perfect working order.
  • devices can be added and removed from nearly any where at any time. If you have a window and there's a power pole or roof access peering into your window, Smile you're on Not Candid Camera
  • it's not a one-off deal
  • go down any street and look UP. look on the buildings. look in the entrance doors. look at the driveways. just stop and look around. It is or will be a future Pokémon Style Game: Find the camera (with video, voice, tracking, telemetry, wifi, voice capture and loads of other goodies).


iirc badly: Some while back, a videographer artist decided to film all the cameras and devices in their neighborhood. It wasn't just a few. No ownership could be determined for an great number of them. They just kept popping up.

AndersSeptember 18, 2019 3:55 AM

What's happening here?
Massive cleaning and post deleting.
Even Snowden lawsuit isn't suitable any more?
@Moderator - any explanation?
Maybe i'm in wrong place and should stop writing
here at all, if anything randomly gets deleted...

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Sidebar photo of Bruce Schneier by Joe MacInnis.