Mr. Pragma • May 31, 2014 10:43 AM

@Nick P, yesme, Clive, Figureitout (and possibly others)

No matter, how I turn it, I arrive at some basic observations and conclusions again and again.

Simplicity, for instance.

RISC rather than CISC, only 1 io bus type, e.g. PCIe or SERDES rather than a plethora, simple languages with a reasonably equipped and sized stdlib rather than “fat” languages (like C++) and fat stdlibs, etc etc.

Clear and concise understanding

We seem to live in a world with a rich variety of solutions – but we actually don’t. Let’s cut through the blabla.
There are, for instance, only 2 architectures of any wide significance, namely IA32/64 and Arm. Yes, there’s Power, Sparc, Mips, and whatnot, too, but those hardly and sparingly or not at all cover the whole range from “Buy it, plug it in, run it” to a variety of boards to a variety of sources and 2nd sources down to a variety of available (also to Joe and Jane Smith) IP, compilers, etc.
As soon as you leave the IA32/64 and Arm world the air gets thin and increasingly expensive.
For the biggest part the impression of a world with with a rich variety of options is propaganda, business interests, and actually to a large degree just copies, me-toos, variants, and “specializations” (lock in).

The “why” leads to “how” and “what”

In a world that was basically unipolar and usa-centric since the beginning of the digital age, the “why” quite usually is business/capital/profit related. Contrary to common belief and quite possibly to the nature of man we are actually not (acting like) curiosity driven inventors. The few who are usually end up either poor or as the 1,5% shareholder of a company monetizing his inventions.
This can also be seen in universities which often are to a shocking degree driven by business and capital interests, too.
Accordingly there are the very centers (like intel) around which, on a global scale, pretty everything is turning, down to me-too companies in e.g. Taiwan which basically, at the lowest end, thrive on delivering a little cheaper.
The amounts involved are so staggeringly high that almost nobody, even not whole nations, can participate. Actually even well-known brand names and billion $ companies themselves, can not afford their own fab. In other words: The whole world basically has to take and to eat whatever very few us corporations please to sell.
FPGAs might look like an alternative. Unfortunately they are not for some reasons, price probably ranking high.

Basically the same core problem, in different disguises and with diverse faces, also creates security problems in other areas like software by brutally favouring “market mechanisms” and profits over sound reason. microsoft, adobe and the like do not produce low quality (and high security risk) because their programmers are bad but because they are driven by criteria that care little or nothing about quality; they pretty much only care about marketing and sales. Features are important, quality and security are not.

Risking to make more enemies than I can count I dare to state that I consider open source to be another major source of problems.
We like (or they like) to see open source as a good thing, almost as a peoples movement against ultracapitalism. Unfortunately we forget that actually a very large part of foss is just another layer around the status quo created by usa corp. interests. Truecrypt is just 1 example; it is centered around windows, one of the major cornerstones of the “corporate/agencies/usa empire”, it has has tu use and respect the windows interfaces, tools, etc.
Another major problem of foss is that some of its core values are not at all related to the matter. To assume, for instance, that democracy generates better code is pure idiocy. What it actually does create is lots of problems and watered down ideas, products, and quality (and hence security).
There are, of course, exceptions, products of high quality developed by professionals and given away for free. But again, these are exceptions. The vast majority of foss code is not only not better than corp. code but actually worse. Anyone doubting that observation should have a look at the openssl code (that, after all, should be assumed to be at the higher end of quality one would think).

Not coincidentially one of the uglier evils in software, bloat, is to be found both in corp. and in foss code. I’m not yet sure, which one is worse, corp bloat, mostly based on insane featuritis, or foss code, based on diverse factors whicg, I think, can be subsumed as sheer idiocy mixed with gross incompetence.

I do have some ideas about solution approaches but those are for another post.

Nice weekend everyone

Mr. Pragma • May 31, 2014 8:44 PM

DB

You know that better than me (I’m neither living in nor travelling to or through the usa) but I’m afraid what you say is unpleasantly often sadly true.

But even (or particularly?) employed developers are in a good position to privately develop good code and share that as (real) open source. Possibly they might sometimes even, to a degree, help others to understand what they work on in their daytime job.

But again, the bad news is, as I’ve written, that foss also created damage in that many companies employ fewer developers and rather rely on free-beer code.

Mr. Pragma • June 1, 2014 12:59 PM

Jacob

Thanks. I know about the Via processors and actually use them for quite some (not too sensitive or performance hungry) applications.

Unlike you (if I get you right) I do not trust taiwanese products; actually I see them as the worst of two worlds because there is considerable chinese influence on taiwan and, way worse, us influence as taiwan happens to be usa oriented, dreaming of usa being their protection power, and focussing much of their economy on usa.

The most trustworthy for me is clearly Russia (because having a strong intelligentsia, excellent academia, and lots of bad experience (with usa) Russians have learned and understood some things), followed (not too closely) by China, and by european countries. Finally, usa products are to be considered the least trustworthy or, to put it less diplomatic and more realistic, as completely rotten. I try to avoid us products for anything even remotely sensitive like the plague.

For common home or sme scenarios, however, via processors (or even us-american stuff) may be a viable alternative provided some prudence re. OS and software.

Mr. Pragma • June 1, 2014 6:54 PM

name.withheld.for.obvious.reasons

As you mention them …

National Semiconductor once also had a very modern and really good processor (series), the NS32x32, the 32732 was, as far as I remember not any more sold but served as an important influence in intels pentium.

Also Siemens (and possibly others) built quite nice systems based on that then very high-tech and modern processor.

As for the Open Hardware Platform …

For a starter I don’t expect too much from google and even if they did something, I wouldn’t trust them and that a microsecond.

And would it be in the interest of fab shops? Not sure. Now with all those Arm thingies the industry probably prefers to not disturb the set up game (intel/arm architectures). After all, don’t forget, in their eyes everything is about shareholder value; they just happen to be in the semiconductor business.
I guess our next real chance are the Chinese. Once they go full scale and global, for instance with their Loongsons, that’ll probably stir up the industry enough for some significant changes. But then, maybe they’ll never do that and rather build up their own eco system with Loongson driven phones, routers, PCs, etc, etc.
Until then we will have to continue using OpenRisc, Lattice Mico, and other soft/firm cores either in fpga or asci. Which isn’t bad at all as long as one isn’t in the high performance area.

Mr. Pragma • June 2, 2014 1:14 PM

Chris

I think there are many misunderstanding regarding random generators.

“Perfection” of distribution, for example, is IMO overrated and misunderstood as a measure of quality.

In the end it’s about unpredictability, i.e. an adversary should not be able to know in advance or in real time about a random series. Granted, that’s a somewhat contorted example but actually you can have perfectly fine encryption with a series of a bln ‘1’ if an adversary has no chance to know that a bln ‘2’ will be/are used.

I think that perfection (perfect equality) of distribution has come into the focus as a major criterion of quality because it is an inherent quality of algorithms that they are predictable. So, obviously, one desires at least a good distribution of random.
Unfortunately, real random (or what is perceived as such by many) is, due to many reasons, very rarely truly random. An example I like to give to students: Assume you are at a very busy pedestrian crossing in Tokyo; moreover assume that 99.99% of adult Japanese people are in between 1m55 and 1m85; also assume that only adult Japanese are allowed to be at that crossing.
Many people think that that would offer good randomness. One could, so they think, install some, say laser based height system at 1,70 and have nice randomness.
An obvious counter example would be (by some coincidence, haha) all Japanese basketball teams having some conference around the corner. But there are more subtle premise errors like, for instance, the — quite frequent — erroneous assumption that the 50/50 of height difference (as compared to 1m70) matches a 50/50 height distribution in the population. Considering that there are also immanent problem factors (like, say, a small quake badly influencing our laser system) it becomes quickly obvious that the matter is more complicated as it might seem.

On the other hand: So what? Suppose the population height distribution is 65/45 rather than 50/50 that is, that we have nor nice equal distribution. As long as our adversary doesn’t know our source for random that’s not a problem. The real problem is the adversary very often knowing our random source, particularly when it’s (math) algorithm based; then it can come down to knowing the seed/start situation.

I personally prefer high frequency event source picking combined with a brutally simple filter to filter out pretty all typical problems like hysteresis, sample jitters, etc.

So the recipe works something like that: Find a source offering events with a very high frequency (radiation, atmospheric noise, etc), pick events with a relatively high frequency and then brutally AND them down and use just 1 bit (the lsb). Of course an adversary might (although very very unlikely) find out about your mechanism, given it’s important valuable enough for him. To do even better you can change a number of parameters like the source frequency, the picking frequency, etc. To do even better, you can use a second random source to control (frequent) changing of parameters. The good thing is that your are hardly limited because your only need is to create unpredictability and not perfect random distribution.

Why this somewhat long elaboration? Well, to be honest because it stroke me one of these days that, given the lousiness of some PRNGS, their shockingly frequent use, and the almost unlimited resources of nsa and similar, one might be reasonably afraid of an PRNG analogue to rainbow tables. Considering that there are, for example, quite some rather simple algorithms in use that, to make it worse, start off with a 16 bit seed, such an attack actually seems not unrealistic.

Mr. Pragma • June 2, 2014 3:24 PM

Anura

Indeed.

But then, using a strictly logical perspective randomness can never be proven because the very act of proving it would constitute a mechanism.

There are many other problems, too, some of them philosophical and while deep (reaching down to the Spinoza question) quite funny.
An example: Assume a trick player were capable of flipping coins seemingly random (out of his control) but actually controlled by him and as he pleases. Then the decision whether any given flip (throwing? pardon my clumsy english) would come to the point of questioning (and proving) his honesty and intention.

Another set of problems arises out of the (highly probable, according to our world view) assumption that we live in a deterministic universe, i.e. that everything happens according to mathematical and physical laws. Which leads us to an funny definition (actually a set of them) of random. I bring this up because it leads us (back) to the (for us) relevant question in that we can reasonably define random as the result of sufficiently complex interactions (and the laws behind it) so as to not be recognizable (and even less synthesizable) to us as non-random.
Which again is funny because this basically states “random ~ complexity” which might make us think again hard about PRNGS.

Happily enough we fell in a trap in (often) confusing intention and means. Random, incl, of course RNGs, is a means in security application; the intention is to deny predictability and pre-cognition and, to a degree delay analysis and breaking (our secrets).
Which, that just as a reminder (for those who always happily blabber “obscurity is no security”), means that RNGs in the end are an important part of a form of obscurity.

Mr. Pragma • June 2, 2014 5:41 PM

Thanks, Nick P

for your constructive feedback. As the usual views on random are pretty well understood and have been discussed ad nauseam – and – as, looking closer, most practically used attacks (concerning crypto) actually do not attack the crypto algorithms per se (but rather e.g. rely on random related weaknesses) I suggest that we follow the approach somewhat more, that does rely less on statistical criteria and qualities.

In concreto I suggest to follow an alternative perspective which focusses the intended goal rather than the means. In other words, in crypto our real intent is not certain statistical qualitites but to deny predictability and analysis (with reasonable time so as to be useful). The fact that statistical criteria and qualities usually are (to be considered) important is not a function of random but our usual way to pseudo-create it, in particular due to the fact that our approach is usually based in a a) rather limited and b) generally available system (to our adversaries, to). Even worse, almost always the adversary has way more resources in terms of that limited approach. In other words, we all use bits and bytes and processors, quite some adversaries, however, have way more resources — and expertise — available than we.
Complexity per se is available to everyone (and be it because it’s inherent in nature). Complexity limited to the realm of computers, however, is by no means available to the same degree for everyone; in fact, often our means are quite low compared to adversaries.
And there are other issues, too. One is that we have a reached a level of “unidirectional” complexity that is so high that Joe and Jane Smith are often almost bound to fail using the means properly. Another one is that complexity has the inherent quality of not being simple (haha) which quite probably (or so experience seems to show) contributes to bad implementations, vulnerabilities, etc.

In this regard I suggest to look at random in terms of complexity rather than in terms of statistical criteria like distribution. But I’d like to limit the view to a “sweet area” where existing complexity can be simply “grabbed” and where a good result isn’t achieved by ever more complex mathematical algorithms (and their complex implementation).

That’s how I arrive at a model where one uses multiple sources “existing” high complexity (rather than creating them) and extremely increase the resulting complexity by mixing those inputs from different sources in well managable (and understood) ways (whereas current solutions tend to simply xor different sources and such waste complexity).

One (intentionally extremely simple) demonstration might be to use source B to determine when and how of source C is mixed into A and when and how much of source D is mixed in. Yet another source E might be used to pre-invert certain parts of A.
Such one could, using per se relatively simple mechanisms to not simply create higher complexity but to also introduce multiple levels of variations, each one quite unpredictable (if a proper source is chosen). As a welcome side effect one might actually use weaknesses currently enabling, for instance, timing attacks.

Mr. Pragma • June 3, 2014 6:10 AM

Now, before someone gets hurt: I intentionally used the coin tossing metapher because it’s quite evident that it’s not really honestly random producing and/or that it’s at least a mechanism with an obvious potential to be bent or tricked.

But that is not even the real problem. The real problem is the (more or less) simple and unidirectional/single mechanism. With coins it’s “Toss it 8 times and you’ve 1 Byte random”. And again, many will immediately recognize and criticise the poor mechanism.
It’s important to understand that we can achieve only actually quite limited gains by enhancing that mechanism, say, by not tossing but by using impressive math algorithms on some seed. Yet, the basic mechanism is the same.
Of course, we are impressed by the immense speed which modern semiconductors can offer. Spitting out gazillions of digits is easily mistaken to indicate a complex mechanism, and there indeed (often) is a complex mechanism – but in the machine used to apply our still simple mechanism.
That quickly becomes dangerous, when
– the machine implementations on which we employ our simple mechanism are very, very, complex and such error prone (read: fertile vulnerability) in themselves.
– the basically simple mechanisms we use are — and must be — so complex that they can be weakened by experts but hardly be understood, and hence often misimplemented, by non-experts (let alone the final users).

Looking at real world attacks suggests that my understanding is not off by far. There are plenty weaknesses, some of them introduced by malevolent parties in the right position, some of them introduced by poor hardware implementations or plain hardware problems in the underlying highly complex hardware (e.g. asic), some of them introduced by poor software implementations or plain problems in the usually fat software stack beneath, some of them introduced by usage errors.

In the end the major difference between coin tossing and modern crypto (real world, not academia) is basically in the complexity of the tosser, of the tossing, and other such factors – but not in the mechanism itself. We’re in a way still tossing coins, albeit with immensely complicated equipment.

Doing this we are the weak party next to one guaranteed to be strong party (cosmos) and one to be assumed (and often rightfully) strong party like the nsa.
Both of them have more atoms, more capability to deal with complications and sheer size, and sheer size themselves than we do or have. In other words: Mother nature is known to have virtually arbitrary complex mechanism to create “random” and nsa and accomplices should be assumed to have more, more advanced, and better means and resources.
So, it seems, largely ignoring one of them and racing against the other one is an undertaking with rather low chances of success.

Hence my proposition to co-operate with the strongest player, mother nature, and to stop creating random but rather to use what existing random is available and to create “managed random” (because that’s what we’re basically after) by mixing and muxing those sources in a relatively simple — but very versatile — mechanism.
After all, we are not after statistically excellent random; that’s just a means. We are looking for unpredictability. We are not working on fair and just lotto machines, we are working on the other side having to pick the right values out of crazyllions of values the order of which they have no chance to know.

Maybe waves are not fairly and justly distributed across any stretch of ocean. But they certainly are highly enough random to deny us any realistic chance to predict them (in fact so much so that we only recently found out that there are quite probably quantum mechanics at work).

Thanks for being patient with me and looking again at random. I’m convinced random, its diverse qualities and mechanisms behind it deserve that (myself possibly not but thanks anyway *g)

Mr. Pragma • June 3, 2014 2:32 PM

juelvo (June 3, 2014 1:14 PM)

Oh yes, those evil despotic Russians!

As we just talk so nicely about evil Russians … it might interest you that quite the same rules are about to be enforced by the “good, democratic, freedom-loving (TM) european union” and are already applied by some euro-countries. In Germany, for instance, some providers are known to keep those data not for 6 but for 12 months.

Never mind, western counties are “good, democratic, freedom-loving (TM)” and those Russian are evil and led by a dictator (who just happens to have way more votes and support in his country than many western leaders combined …)

@anonymous ass (June 3, 2014 2:02 PM)

If that makes me think or assume then I’d rather assume that those issues were factors for Bruce Schneier to leave brit t’com.

But, of course, implying that he, too, is a part of the evil side, makes you sound so much more interesting and smart.

Mr. Pragma • June 4, 2014 9:41 AM

Jacob (June 4, 2014 9:25 AM)

Yes, right!

obama sounds like a guy telling you “I fu**ed your wife, cut your tires, abused your daughter. What a shame that some evil guy told you about all that such straining our good relationship”.

Not quite as bad but similarly telling is the Germany governments reaction. Basically they confess “yes, it’s true; we are but a usa vassal and colony. Too bad that certain events disturbed our ‘independant, free, and democratic country simulation'”.

@juelvo

try something smarter. Meanwhile nuland yourself.

Mr. Pragma • June 4, 2014 12:54 PM

Moderator

What is it that you find unpleasant? The term “to nuland” or its usage and the perceived insult?
If it’s the term you should resolve that with the (self censored) in your state department.
If it’s the perceived insult you might want to consider changing your perspective that finds the request “nuland yourself” inacceptable and insulting but has no qualms whatsoever about other users insulting and ridiculing a whole people and, along the way, many victims of usa-sponsored terrorism leading to children and hospitals being shelled.

And no, I’m usually not debating a moderator. When you gave me hint recently, I obliged and stayed away from a matter that you wanted quietly passing away.

And btw maybe to nuland oneself means to treat oneself with cookies. Obviously my intention was not to insult but to show utter dismay; otherwise I would have used the word that you felt to be implied.