Schneier on Security

Clive Robinson • October 31, 2020 7:22 AM

@ SpaceLifeForm,

Or do you want to believe the tales of a mannequin?

Yes… “speak no evil, hear no evil” as we know “crash test dummies”, “mannequins”, and “Martinets” are not the talkative types they are too “stiff necked” let alone “stiff upper lipped” and as others would observe “Lacking the I in AI and Id”.

The article on the other hand is probably taking a few liberties with syaing this,

“Infectious-disease and healthcare experts say the study correctly concludes that the infection risk is lower on a plane than in places such as stores and restaurants.”

You note they name people they publish the opinions of, yet are strangly coy on this opinion…

I realy can not see any reputable infectious disease or healthcare expert saying that. It’s an apples to oranges situation and also unmeasurable in any meaningfull way…

After all if you are the only person sitting in a restaurant with your mask off eating where the restaurant, that takes reasonable precautions then your infection risk is probably less than walking down a quiet street without a mask on…

The simple fact is you can bet for the “tests” they did not just pull an aircraft straight out of service and use it… You can bet a specialist maintaince team went over it very very carefully and ensured everything was brand new and airtight etc. So what’s the betting any aircraft you get on has had the same level of attention? Yup I wouldn’t bet on it either…

Yes airlines are going to go out of business, yes flights are going to get expensive, but no it realy does not matter it’s just another “end of life” “service sector” clutching at straws…

After all why spend half a day to a day getting to the airport, going through checkin security etc etc, spend hours on what is a healthrisk above the norm (remember DVT etc), just to arive somewhere to spend 14days imprisoned in an overpriced hotel with a rent-a-cop on the door where one or more of them are looking for a few extras from the inmates…

It might be better to hop on a “slow boat” and swap a “flying petri dish” for a floating one[1].

The simple fact is like it or not ICT has reached the point where we can have high resolution near enough real time communications to most places we might go to visit.

Back three decades ago a Reader at a University I was attending posed a question,

“When the bandwidth and technology is sufficient to make it feel like you are there, but still with all the comforts of home, why fly half way around the world to kiss Granny on the cheek?”

I suspect it’s a question quite a few more people will be asking themselves, especially with two weeks of quarantine when you get there and two weeks of quarantine when you get back…

Four weeks of tedious isolation ontop of two days or more of unpleasant travel is a lot to ask, just “to press the flesh”.

[1] Personally I don’t even get on trains or busses now due to the fact the Drs have told me my risk factor is worse than that of a blind drunk motorcyclist having their first try at the wall of death…

Clive Robinson • November 1, 2020 3:06 AM

@ Bob Paddock,

I find myself in need of such things, unfortunately, are there any that have non-pathetic security?

It rather depends on your definition of security and where you want it to start and end.

CCTV cameras –which is what nany cams are– come in a variaty of classes.

Firstly there is the analog / digital divide. Whilst analog is tending to disappear they are still around and are generally easier to secure when using dedicated wiring.

Which brings us to the next division, wired / wireless. Whilst wireless in the ISM bands is all the rage in consumer devices it’s laughed at by the brighter criminals and avoided by proffesionals. Because it’s at the very least a “beacon” giving away not just it’s presence but it’s location, and it does not matter if it’s analog, digital, or encrypted the systems are usually insecure in that the signal changes as things apear in it’s field of view, thus a smarter criminal can work out not just it’s field of view but it’s range at night as well as during the day.

But the digital side of things has several sub domains. Many “office cameras” are IP over Ethetnet on Cat5 or above cabling with PoE being not uncommon. Whilst Ethetnet protocols are not routable, IP protocols are, and that creates a whole bunch more security issues. It also creates other problems, such as information bandwidth. Many of the more sensibly designed IP based cameras use methods to reduce data transmission. Which means that the network traffic is quite variable. Thus it is possible to set systems up that work fine under normal conditions but fail in the likes of fire alarms when suddenly all camaras are maxed out.

But as I’ve remarked before the market place has changed and the race to the bottom marketing techniques have actually punched through ground zero and are heading ever downwards draging the rest of us to hell and beyond. In essence these IoT type cameras have minimal cost components, and all the features are done over the Internet where extra money is made. This could be by paying “cost or less” for the hardware but then having to rent by subscription etc the service to use the hardware you have purchased but now do not actually own… And paying for the service leaks all sorts of PPI that can then be “repackaged, wrapped up and sold” to third parties you have no relationship with at all. Worse if the company folds or decides they won’t support your cameras any more, they stop working… This is known by some as “Doing an Amazon”.

But speaking of Amazon their Ring system is realy realy bad news from an information privacy viewpoint. It turns out not only are they insecure, Amazon also gives your feeds to LEO’s without any form of privacy protection…

I hope that helps in a general non product specific way.

But if you want a more specific way, there is an Australian company Swann whose miniture wireless products I’ve put in model railway engines to give people “drivers eye” and “pasenger eye” views whilst playing with them.

Thus have you considered making your own “nanny-cam”?

Because I’ve also attached them to RC aircraft, helicopters cars and boats, as they are generally a lot less expensive than “drone” equivalents.

I’ve also used their wired cameras which you can put unobtrusively in objects such as clocks, fake fire detectors, alarm detectors, box files, lighting fitings and even computers etc. The ones I used were not “IP based” so are not a remote security risk and their local risk is more based on your abilities to hide them in objects and get the cables away.

For some reason their US and UK web site does not show up the cameras alone, just finished quite obtrusive security cameras.

So you might have to use other suppliers via China and AliBaba etc.

Clive Robinson • November 1, 2020 3:54 AM

@ MarkH,

To be OTP by Shannon’s definition, the keystream must be truly random

Actually no, the proof does not require “truly random”, it only requires it to be “non determanistic” to the observer such that “all messages are equiprobable”.

The difference is subtle but “truly random” can be shown to be insecure for a number of reasons as I’ve explained some of in the past.

@ FA,

You should get your terminology right. Whatever is generated and can be regenerated from a shorter input is definitely NOT an OTP.

That is something that is not exactly true.

Party A generates a very large nondetermanistic file that they give to party B. Which is a standard OTP procedure.

What party A does to encrypt a message is use a part of that file that has not been used thus it is an OTP encryption.

In normal OTP usage some kind of index into the file or message indicator is sent which can be sent with the OTP or sent separately from the OTP.

Thus a short “index” will regenerate the required OTP from the file, as securely as an OTP.

@ SpaceLifeForm,

Yes you may have a weakness in the system,

Alice generates a OTP that is of the same size of the chunk of data which includes the signature.

If I am reading your “shortform” correctly then your plaintext going into the OTP is,

Plaintext = [data + signiture]

Where “+” implies a cancatonation or other linking process in some way.

If this is the case then it breaks the OTP “equiprobable” rule because it provides a method of distinguishing a valid message from many many other invalid messages.

It’s the reason you should not compress the plaintext input to an OTP (especially using any of the normal compression systems).

It’s just one of those subtle things about OTP’s that don’t get into the text books.

Clive Robinson • November 1, 2020 6:13 AM

@ name.withheld…,

Where the FEC has had a clear role in traditional media…

Two pertinent questions at this point is “Who runs the FEC?” and “How did they get their positions?”

One of the things I forgot to mention to @vas pup, the other day is “It’s not just money, that needs removing from the equation, it’s “appointees to.”.

But also for the sake of society lower apointments such as are made by “people voting” is not a good idea either, there are way to many who get “voted in” who run things without impartiality, or fairly in other ways.

It is easy after all to pull all kinds of questionable activities when oversight etc is also elected, thus you have whole Dept’s where “crime is part of the benifits”. Even Judges have been found getting what we might call “Election Help” by privatised organisations such as Prisons and those providing uncompetative but highly profitable services to not just Prisoners but others trapped in systems run for profit.

It’s the flip side of the “Great American Dream” when the resources run out or become –deliberatly scarces[1]– then others profit greatly especially if they can become a monopoly in some way.

[1] A classic example a few years back when a certain financial organisation got hold of the aluminium supply the profit they made by just stacking the ingots up in yards under tarpaulins was eye wateringly large. But whilst just about every consumer in the US payed this “tax” those who effectively ran an illegal monopoly hardly got mentioned or punished…

Clive Robinson • November 2, 2020 5:11 AM

Sancho_P,

This is something I do not understand – could you elaborate, please?

Yes, but first you have to remember is that,

“To err is human”

That is all systems are subject to human failings, even at the best of times.

Secondly you have to remember,

“The Enemy Knows The System[1]”

Is quite a bit broader than just the crypto algorithm or mode it is used in. In essence it means that the enemy knows not just about the whole communications system but the probable content of the messages as well (which gives rise to “known plaintext” as well as “Traffic Analysis” attacks).

So any compression system used will be known to the attacker. As I indicated most compression systems work on the message in a way that acts as a distinquisher. That is many compression systems have an internal structure that can be verified as correct or in-error[2]. If you look at all messages of the same length as the compressed file, by far the majority will not produce a valid output when put through decompression.

So not only have you eliminated by far the majority of messages, you’ve also broken the “OTP proof” of “all messages are equiprobable”.

Whilst this does not eliminate all methods of compression you have to find those that maintain or keep close to “equiprobable” that is do not add structure or distinquishers to the message.

But this raises the question with other cipher types including stream ciphers of “Why do people talk about using compression?”

Well it’s to do with statistics. You hear of “flatening the statistics of the plaintext” that is reducing or removing natural language or other recognisable features of the message. In essense trying to make the message unrecognisable statistically. Which does work with the right kind of compression, that is if it also acts like a form of encryption.

This was a necessary step with older cipher methods that could be attacked through various statistical methods. For instance simple substitution or transposition at the language alphabet level did not change the letter frequency statistics. Even combining them and repeating several steps of substitution and transposition did not change them as all the substitution mapings compress into a single mapping likewise the transposition mappings.

But with encryption algorithms of sufficient quality used in the right type of modes it should be unnecessary.

However using compression does alow you to change the encrypted message length, which for years was also desirable because reducing the size of a message had other benifits, not the least of which was cost, which is why the later Victorians were apparently “code book crazy”.

If you are looking to use compression appart from using one that has low artifacts in it’s output, try and find one that has variable “fractionation”[3] properties, which also provides strong “diffusion” within the message. Which is one of Claude Shannon’s primary requirments of an encryption “system”[4] along with “confusion” brought about with the mixing with the key.

Many get the incorrect assumption that diffusion and confusion” are ment to be part of the encryption algorithm which is why you get statments such as “diffusion is not a property of stream ciphers only block ciphers”. It is actually only true dependant on your definition of a stream cipher and a block cipher[5] which tends to be muddled depending on the level you look at them and if they fractionate or not causing diffusion.

So with a true stream cipher that does not cause diffusion, the preprocess of compression if propperly done does cause diffusion in the message. Leaving the confusion to the mixing process.

The way most OTP’s in pencil and paper form are used they are actually block ciphers where the block is the size of the message underlying alphabet, unless the alphabet size is changed (as in Number OTP’s not Letter OTP’s). Whilst this does add some limited diffusion it’s weak, but the nature of the key stream being non determanistic so overwhelms this with the strongest of confusion it makes little difference.

But if you are going to use OTP systems, I’d give the advice that you use it “Only as super-encryption” that is you actually encrypt the message with a good encryption algorithm and mode before you push it into the OTP. For two reasons,

1, It will significantly limit the damage done if due to error you get OTP “key-reuse”.

2, It will flatten the statistics rather better than most compression algorithms.

Thus you avoid not just the “to err is human” failing but weak fractionation that Project VENONA[6] used over four decades to reveal Soviet Spys etc.

But importantly this way you can use compression to change the message length prior to the use of the good encryption algorithm. The point being you have seperated the changing of the message length and message statistics flattening, thus can optomise them seperately, as well as change them easily when required.

[1] Is actually a quote from Claude Shannon, which restates Auguste Kerckhoff’s second principle of cipher systems,

“It should not require secrecy, and it should not be a problem if it falls into enemy hands;”

In a broader setting than just the cipher part of the overall system in use.

[2] If you look at how many compression software systems work, the output file has a structure, which is in part input file (message) specific. That is a dynamic dictionary is built as part of it. This structure is recognisable in the output file. Such structure will not be visable in by far the majority of possible message texts of that length.

[3] https://en.m.wikipedia.org/wiki/Transposition_cipher#Fractionation

[4] https://en.m.wikipedia.org/wiki/Confusion_and_diffusion#Theory

[5] The misunderstanding exists because of loose definitions, for instance the mixing function in a “stream cipher” can be by either “bit based addition mod two” (bit level XOR) or by a larger block such as a byte or word thus “block addition mod block size” (such as byte or word addition without carry). They are both “seen as stream ciphers” due to their similarities but they are not which can be seen by how the mixing function behaves. Whilst the bit addition by XOR does not cause diffusion as it does not carry across bits, the byte or word addition does as the carry process can cause all bits in the block to carry into the bits above it, thus it “fractionates”.

[6] https://en.m.wikipedia.org/wiki/Venona_project#Decryption

Clive Robinson • November 2, 2020 5:41 AM

@ SpaceLifeForm,

As, to the “equiprobable” issue:
There must be a signature otherwise any random garbage can be decrypted to better smelling random garbage. So, maybe need to append some random garbage.

The important thing is to stop the signature becoming a distinquisher.

One solution is to modify the signature in a way that makes it “equiprobable” as well.

One way to do this is to add a secret IV to the start of the message hash process that goes into the signature. Then random fill the rest of the available space to fill the input to the ECC signing process.

I hope that helps.

Clive Robinson • November 2, 2020 2:53 PM

@ FA,

I don’t buy this. There is no rule that says that the input to OTP encryption must not have redundancy.

I never said their was, to remove redundancy is to remove information capacity which is generally kind of self defeating.

However there are occasions when it is done, and for good reason, that is for a specific set of advantages, such as for “error checking / correction” and similar.

So if you look at a simple case of a checksum, the,

Message = [data + Checksum]

form a unique pair[1].

Change either the data or the checksum in any way, and the unique pairing created by the checksum algorithm is not just broken it is easily detectable as such. Which is the purpose of digital signitures when you think about it.

So if I get given all the possible messages for a given message length how many are going to fall into having that unique pairing and how many are not?

Have a think on it, and the implications.

[1] This assumes that the “data” is of a size that does not cause a check sum overflow, which is usually the case with modern hash functions used in digital signatures.

Clive Robinson • November 2, 2020 5:11 PM

@ MarkH, FA,

But how does that help an adversary?

Firstly you need to ask who your “adverary” is. So you need to consider what a very well resourced Level III could do.

In theory with properly designed and operated systems the answer is nothing even with known plaintext.

But as I’ve said,

“To err is human”

The major failing of OTP’s that you get told about is “operator error” giving rise to “keystream reuse”. Less often mentioned is the system design around the OTP is a compleate screwup.

How you take advantage of these “err’s” as an adversary is rather boringly mundane. But can be very worthwhile which is what the four decades of Project VENONA by the NSA and GCHQ was all about.

So in the case of suspected keystream reuse or failure in KeyMat production you compare OTP ciphertexts to strip off the keystream. You are looking for a way to distinguish when the key-streams are not just the same but also aligned. This is usually done by looking at the statistics of the result.

Thus you need to design your system such that it does not produce changes in the statistics or other distinguishers.

But there are other problems unless the plaintext into the OTP’s is “recognisable” in some way you can end up with a lot of probables. In the case of Project VENONA they had to deal with how the Soviet OTP System changed the alphabet size, which acted like a low grade cipher.

So as I indicated you should not compress but encrypt to flatten the statistics. Depending on the quality of the encryption algoritm and how you implement it, this obviously significantly increases the “probables”. If done right –which it rarely is– you would have too little statistical information thus the number of the probables increases to the point where they become unmanageable unless there is some kind of easy to use distinquisher which is what a checksum or digital signiture is.

The simplest way to avoid this checksum problem is to do the OTP encryption and then do a checksum or hash[1] of the ciphertext not the plaintext.

Anyway as I’ve said go and have a think on this as your questions suggest you’ve not had reason to before.

Think especially in the terms of what a National Sig Int agency etc can bring to focus on any “operator errors” that can reveal “design errors”.

There are many reasons why OTP’s are not practical or safe to use and few people ever discuss them such as the issues involving KeyMat generation. Frequently those that do the KeyMat side of things are given “instructions to follow” and are never told the reason why the instructions are what they are. Hence things become a “witches brew of incantations” and other “dark arts” and “ju ju magic”. Similar on the operator side and those who write about OTP’s. Hence the usage of OTP’s get an undeserved aurora of invulnerability they realy do not deserve and just about any programmer thinks at some point they can design an invulnerable crypto system around them…

[1] Whilst a hash is fine it’s actually unwise to sign it as that ties it to a PKI cert, which if known by the adversary is effectively pointing a gun at your head. If you must use signitures then you need some way to make them unusable to anyone but the intended recipient and only identifiable to single messages Thus the use of a secret message specific IV into the hash or similar and random padding to fill the rest of the signature input.

Clive Robinson • November 2, 2020 6:29 PM

@ Thunderbird,

“if no system mistakes occur…

Yes, but whilst “if” is a small word it has big implications…

As has become clear over the years on this blog nearly every one has heard of OTP’s and “Perfect Secrecy” but from then on in their knowledge gets patchy.

So whilst quite a few know about,

1, The KeyMat has to be as big as the message.

2, The KeyMat must never be reused.

You should ask have they thought about the implications of the points when it comes to generating KeyMat?

In most cases probably not. So they are probably compleatly unaware of why bounds need to be put on “truly random” to reduce not just information leakage, but inadvertant in KeyMat key-reuse. Oh and have thet thought further that the bounds mean that the amount of KeyMat that can be generated is also bounded thus finite (which has major implications in large scale networks).

But how about a simpler question, how many are aware that in the common use of paper and pad OTP’s the mixing function is “addative over a small block of bits” not as is often taught a bitwise XOR?

Even if they are, are they aware of how this effects “diffusion”?

Probably not, so when using the mixing function as a building block in a system will they evaluate it correctly?

The list goes “on and on” and each time I mention one the response is in effect “but… Perfect Secrecy”.

It’s why I hardly ever use the term prefering to state the actuall security proof which is “all messages of the same length are equiprobable”.

That way you can see that an OTP with a very short message of say just two characters, the probability is the last one is a Carriage Return(CR) or Line Feed(LF) and is very likely to be preceded by a Y or N or single digit number or one of the first few letters of the alphabet (ie menu system selection). Which reveals a part of the KeyMat, which if imperfectly generated alows for the possibility of attack by an adversary. Thus if it’s used with a fixed format the adverary knows the plaintext for… So an OTP system realy should not be used with interactive systems such as a computer terminal, where known plaintext abounds, nor the likes of Word Processer files where upto the first 4k can be “known plaintext” similarly other common file formats. Just having “probable KeyMat” of such length makes attacking systems where key-reuse happens due to poor KeyMat generation or poor system design, oh so much easier…

Bob Morris senior, on retiring from his job at the NSA gave an open speach, in which he warned about known plaintex… Something people who design systems realy realy should keep in their head.

The big problem is not designing an OTP system, but due to the fragility of the OTP designing a system that remains secure in the face of a multitude of potential failure modes, and very very few can do that, even when they know what the failure modes are (ie prevention of key-reuse due to poor KeyMan).

Clive Robinson • November 3, 2020 4:34 AM

@ FA,

Now please explain what, according to you reasoning, this reveals about D.

It tells Eve if the plaintext (Dn+Cn) (where n denotes bit length) is valid or not thus acts as a distinquisher to 1/(2^Cn), therby eliminating 1-(1/(2^Cn)) of the potential equiprobable plaintexts of length Dn.

That is what you would expect of a hash that does not have collisions with messages of length Dn.

Clive Robinson • November 3, 2020 5:17 AM

@ Winter,

It would seem to me that if there are such puppet masters, they would not show up at Davos.

They do not, it’s their higher level cut outs throwing out “groundbait”[1] to lure towards the hook on the end of puppet masters line. If you are smart you see the groundbait for what it is and stear clear.

As for “winowing out the population” in the UK just about every step taken by the Government encorraged the deaths of the old and infirm. In Spain the triage indtructions in hospitals was not to put the retired on ventilation, and in some cases deliberately remove ventillation if they were on it so that the “economically viable” were prioritised.

But as was said at the very begining,

1, Close the borders.
2, Stop / limit flights.
3, Isolate symptomatic in healthcare provision.
4, Institute drive in testing remotely from healthcare.
5, Institute compulsory hard quarantine.
6, Ensure mask availability.
7, Implement track and trace.

Was their priority order for those countries that have been hardly touched by COVID. Because they understood the nature of exponential rise and the importance of clamping down early, to protect not just the economy but the citizens.

The US and UK very clearly pandered to the short term interests of the transport and hospitality industries, and only grudgingly took action after the number of cases were already compleatly out of hand.

Then, rather than implement the priorities in the sensible order they took the reverse order of least effective measures first… Abd in track and trace botched it up so many times it was a compleat comedy of errors that had no humour in it what so ever.

The evidence so far is stacking up against your position, and I sadly suspect it will continue to do so, as I’ve indicated the current education policy is to turn children into granny killers. Not exactly a humane thing to do for the elderly who will die in a horible way, or for the children who later in life will realise their inadvertent guilt, thus develop a hate and loathing for the neo-con behaviours that turned them into “child soldiers” of a sick economic war.

[1] Not sure what your native language equivalent of “groundbait” is. But it’s the equivalent of shark fishers “chum” for fresh water fishing. That is it is “cast upon the water” to “lure not land” the prey. In essence it pulls in the less smart prey one of which will gobble the bait on the hook and thus can be landed and dropped in the “keep net” trapped untill dispatched or set free.

Clive Robinson • November 3, 2020 6:06 AM

@ FA,

I’m trying to grok what you mean here, but I fail.

As I’ve indicated with “to err is human” and “the eneny knows the system” the adversary has either KeyMat or Known Plaintext they can use to find KeyMat.

This assumed KeyMat is then used to try and find “key-reuse” which produces a very large number of possible plaintext messages.

Because having a copy of some or all of the KeyMat does not enable an adversary to use it.

That is the adversary has to know which section of the KeyMat is used for any given piece of ciphertext. If the users of the system have a way of hiding this behind “key indicators” that are effectively hidden (ie an index into a shared lookup table the adversary does not have) then a brut force –sliding rod– search has to be carried out by the adversary.

Which means the adversary ends up with as many probable plaintexts as there are bits in the KeyMat the adversary has.

The problem the adversary then has is finding which of the probable plaintexts they have generated is the correct message –if there is one– from amongst all those probable plaintexts the adversary has created.

There are just two ways to do this,

1, By statistical means.
2, By identifying distinquisher.

Of the two an identifying distinquisher is by far the best.

So when you design a system around an OTP you try your best to design out any distinquishers or usable statistics.

The use of most modern compression systems provide both weak distinquishers and supprisingly to some weak but more than usable statistics.

Hence you should only use an OTP as a form of super-encryption, after you have already appled a suitably strong form of statistics flattening and distinquisher removing encryption process that also has wide ranging fractionation / diffusion charecteristics across the message length.

The use of a strong distinquisher such as a checksum, hash, or signiture inside the OTP encrypted section of the communications should be strongly resisted by the designer. If there are technical reasons why this can not be done, then the distinquisher has to be strongly mitigated by a secure method unique not just to the communicating parties but at the individual message level.

Because at the end of the day two things can be said,

1, The users will “err”.
2, A Level III adversary will record all communications.

Thus it’s not just the whole system “that the enemy knows” it’s “every err that has been made” as well.

Thus the system designers job in not just to get a working OTP based system, but one secure against any and all “errs” the operators of the system might make. Which includes the theft of KeyMat and “Known Plaintext” or even “Probable Plaintext” based on related plaintext and “gardening” / “traffic analysis”[1] due to use of standard formats and the like.

[1] Many of these techniques were used to get “probable plaintext” during WWII to form “cribs” used on the bombs at Bletchly and later in the US. Two types of “gardening” are documented, one was the use of dropping mines from aircraft, such that a “standard worded” warning message would be sent on the “dockyard cipher” Which coukd be broken, this then gave a “known plaintext” message that would be sent out over the Donitz’s U-Boat Fleet Broafcast system, thus enabaling the daily four wheel Enigma key to be found. Another was to put contentious columns in news papers knowing that the embassy staff would put the column in a high level cipher back to their soverign state “word for word” in the current key used for all messages in that period…

Clive Robinson • November 3, 2020 8:55 AM

@ rrd,

When someone’s ego is completely dominant, their only purpose is to give themselves a dopamine bump by projecting superiority upon others,

Oh dear more of your projecting your at best odd failings out into the universe.

When are you actuall going to realise your failings and take steps to address them?

Clive Robinson • November 3, 2020 12:41 PM

@ Winter,

So, If I understand it correctly, OTP works well if, and only if, you use real random key material and every bit is only used once.

Not quite “real random” has to be constrained or bounded otherwise there is the possibility of plaintext leakage, but also inadvertent key-reuse, the probability of which gets worse the more you constrain the KeyMat (so cursed if you do, damned if you don’t).

But there is a more subtle problem, which is as a cipher it offers only Shannon Confusion, not Shannon Diffusion, which makes the OTP system brittle / fragile.

Thus appart from generating real pencil and paper One Time Pads for the likes of emergancy re-keying after the loss of KeyMat for other cipher systems or for giving super-encryption, it does not have any real practical use.

But worse the KeyMan from generation of the KeyMat, auditing, transportation, storage and disposal due to the vast quantity of KeyMat involved is just a nightmare.

As our host @Bruce noted not long after a NIST Crypto Competition, we need to move on to dealing with Key Managment because it’s an area that is very neglected not just in academia but in the Open Information arena (as anyone doing a depth or breadth search will find).

Our current solution to KeyMan by PKI is to put it mildly “a disaster area that has happened”, with people talking about certs being 8k bits or more to be secure, that’s way longer than a half dozen consecutive SMS messages or original tweets.

Compared to OTP KeyMan PKI would appear to be a positive delight…

Yes there are niche uses for OTP’s but you realy want to avoid them if and where ever you can.

And before you ask yes I do talk about using them as an example of a secure pencil and paper cipher, that can be moderately easy to use. But it is more as an example for the purposes of explanation rather than use some mainly non existant secure electronic token (with all the EmSec issues on top). That is people rather more intuatively understand the risks involved with pencil and paper than some electronic gizmo.

If you look back far enough in this blog you will find discussions between @Nick P and myself about not just “chaining ciphers” but also hybrid ciphers. In there was a discussion of taking a block cipher “mixer function” using Fiestel type rounds and replacing the key schedual functions that generate the rounds keys with either a shift register feed from a “stream generator” or via several stream generators each one for a subset of the rounds.

The point is the rounds provide the Shannon Diffusion along with some Shannon Confusion.

Clive Robinson • November 4, 2020 1:05 PM

@ FA,

What is the difference ?

About the same as one combination lock with 10^6 wheels and 10^6 locks of one wheel each.

As for,

That is irrelevant to this discussion, and you know it.

Actually no it is highly relevant and yes I do no it, which you apparently do not.

I suggest you contemplate what happens when things go wrong with TRBG’s, how you recognize the fact they have gone wrong and how, and what you do about it.

As a designer of systems you have to live in the real world where things always go wrong at some point and what you do to mitigate thst practical reality.

Clive Robinson • November 4, 2020 2:45 PM

@ rrd, ALL,

How do multi-bit functions for combining chunks of message and OTP compare to XOR?

I’ve already answered this, but I will say it again.

The mixing function can work on just one bit (XOR) or many bit blocks / “chunks” (ADD etc) the important question is what happens when you change an input bit? how many output bits change?

For both Shannon Diffusion and Shannon Confusion you would like on average half the bits in the output to change for just one bit of change at the input.

In a true OTP where you XOR on a single bit there is no Shannon Diffusion into other bits, and Shannon Confusion is limited to a single bit. This is true even when you XOR a byte, word or larger block / chunk. Because the XOR function is a “bitwide or logical operator”. If however you use say ADD whilst the least significant bit is still XOR, subsequent bits have the overflow from previous bits affecting them as “carry” these are known as “bytewide or arithmetic operators”. Which can provide both Shannon Diffusion and Shannon Confusion.

So you would need to analyze the mixing function on a case by case basis. For instance if you multiply (MUL) rather than use addition (ADD) the least significan bit is actually the AND function not the XOR function.

These least significant bit oddities of XOR for ADD and AND for MUL, were taught to you in primary school with the rules for working with even(E) and odd(O) numbers,

E+E=E, E+O=O, O+E=O, O+O=E

EE=E, EO=O, OE=O, OO=O

If you draw these up as logical truth tables you will see they corespond to XOR and AND respectively.

The same rules actually apply to all bits in the results if they have not had a “carry in”[1]. But looking only at the output without either input you can only get some “probable” information from the least significan bits of the MUL as the AND function is unbalanced in it’s output thus leaks some information about it’s inputs –both odd,– that might or might not be useful in cryptanalysis.

[1] If you doubt this then go and look up the logic diagrams for hardware Adders and Multipliers. A search on “ALU” or “bit slice processor” with “diagram” usually pulls up the required information,

https://en.m.wikipedia.org/wiki/File:Binary_multi1.jpg

Shows the four AND gate array used for bit multiplying followed by two full adders to get the required four bit result. You can see from this that the LSBit C0 is just a single AND gate and the full adders use XOR gates for output and AND for carry.

Clive Robinson • November 4, 2020 3:16 PM

@ MarkH,

For practical purposes, we can call a high-quality hardware generator a TRNG … when it’s working correctly. But when it’s malfunctioning, it is not a TRNG any longer!

Hmm not sure how many people would agree with you.

Look at it this way,

You are driving in your car and it stalls and stops and you can not start it. When you call the AutoClub to get a repair man, do you say “can you come and fix my car” or do you say “can you fix my mechanical contrivance that nolonger functions”.

Likrwise does your pocket radio cease to be a pocket radio when it has a dead battery in it?

A TRBG is still a TRBG irrespective of if it is working or not, it is upto the user to decide if it is functioning or not hence people say “My XXXX is working” or “My XXXX is not working”, or “My XXXX is broken”…

That is the name of something rarely changes if it’s usage state changes.

Clive Robinson • November 5, 2020 12:27 PM

@ Sherman Jay,

Credentials are not automatic indicators of authority.

They should not be, like respect credibility needs to be developed or earned depending on your chosen phraseology.

However for reasons best known to themselves our legal breatheren deal mainly not in the real world but a poor facsimile of it based on words on piles of paper. Thus they more or lrs give credibility based on what pirces of paper give you “standing” in their eyes.

Having had to go through the process in the past, I’m aware it is a “sword that is three edged”.

Clive Robinson • November 5, 2020 4:06 PM

@ MarkH,

As far as I’m aware, in the U.S. legal certification for engineers is limited to expertise needed for large boilers and other pressure vessels.

It depends on your definition of “engineer” and how you view people who design and build houses, buildings, ships, planes and automobiles.

But it all goes back to,

“First Do No Harm”

From some acient greek bloke 😉

Oh and the slightly more modern,

Q: What do chefs do with sauces, lawyers with words, and doctors with dirt?

A: Cover up their mistakes.

But it also involves the notion of “Guilds” etc where “artisans” clubed together to come up with “trade secrets” and “cheap labour” and cartels to increase profits.

The result has been the “legal professions” by which I do not mean Lawyers but any group of people including accountants and doctors that have the legal right to give and take away “a licence to practice”.

The argument is that certified proffesionals ensure safe, honest and trustworthy work is carried out. As we should know that is far from true, in fact proffesions can actually hide unsafe, dishonest and thoroughly untrustworthy individuals (Look up Dr Harold Shipman or Nurse Beverley Allitt)

As for “pressure boilers” the requirment came about in the Victorian era with boiler explosions flinging body parts around the scenery. In the UK Parliment enacted legislation that made a distinction between artisan’s such as “wheel-wrights” and boiler “engineers”.

In Europe in quite a few countries you have to be a Registered Technician or Chartered Engineer to carry out many types of work and if you register with the right body you are entitled to put T.Eng or C.Eng after your name so you could end up with,

“Dr Fred Smith C.Eng, T.Eng, PhD, MSc, BSc”

Etc but I for one don’t bother it’s two pretentious. Also if you go around calling yourself Dr you know darn well you will be at some social event and somebody will take you aside and confidentially ask you about their bad back[1] or worse. Then it gets realy embarising trying to explain you are not that sort of doctor (especially if you are a DD or worse yet a “Doctor of Sacred Theology” which is “STD”, who wants to add that to their name ;-).

[1] I have a standard reply for those who have the cheek to ask me about their back problems which is “Have you had a dentist check you out?” and if that fails “How about an opticion or podiatrist?”. Before you say “but but but… your not qualified” I’m not giving opinion or advice just asking questions. But it turns out for those that want to go look it up, current thinking is most back problems are caused by what is politely called “poor posture” and the only cure is to correct it. So squinting, clenching your teeth, and different length legs are often the root cause of bad backs, and the cure is get the posture problem sorted out and spend time with a physiotherapist to learn the right excercises to undo the muscle problems the poor posture has caused. Unfortunately there are some, myself included that have other issues for their bad backs that cannot be so easily corrected if at all, hence my getting around on crutches which does work upto a point, rather than have spinal injections or operations that something like 9 times out of ten don’t work…

Clive Robinson • November 5, 2020 5:38 PM

@ JonKnowsNothing,

That and learn how to eat a whole raw peach with a knife and fork.

Skined or unskined 😉

The secret to doing it for those who have not tried is a fork with very small tines and a very sharp knife. The first cut is to slice a flat down to the stone then turn the whole peach onto the flat… If it’s realy ripe you can be flashy and make an equitorial cut, halve lift out the stone turn both halves cut side down and proceed from there…

The thing I never got the hang of was eating “milly-fillies” or more corectly “mille-feuille” which are simply a sandwich of fine puff pastry seperated about an inch and a half appart with “Crème Pâtissière” and sometimes fruit as well, the top of the top slice glazed with a layer of white sugar icing and on this thinnly piped lines of chocolate (solid choc not a sauce). The slice is about two inches wide and four long, and has verticle cut sides…

It’s hard enough making the darn things without botching it up, but when the fruit is halved glacier cherries or segment of poached pears or peaches the fun with eating the darn thing without making a mess realy starts. I’ve only seen three people do this with “elegance” one was a PhD student from the Far East who also had a taste for raw garlic and ginger, both sliced so thin they were translucent and a great acompliment to raw oily fish or as a snack on their own, she would effortlessly pick individual slices up with chop sticks, drop it delicately on her tounge and savor slowly… She did it with mesmerizing elegance and if she caught you watching she would get all embarrassed and giggle slightly.

Though I’ve been using chopsticks very regularly for over four decades and can pick up individual grains of rice fairly confidently and baked beans without them flying, I still can not pick up a thin slice of garlic and it’s frustrating frustrating frustraiting 🙁

But whilst I still will not eat a milly-filly in company, I’m happy enough making them and inflicting them on others with a fine sliced wild strawberry spread in a rosette on a fine square slice of bitter chocolate and a very light dusting of icing sugar and a couple of baby mint leaves.

Or if I’m fealing realy mean, a verticle “fruit cornucopia” made from either puff pastry or piped meringue…

You can fight many a battle with fine patisserie, so my advice is stick to the cheese board 😉

Clive Robinson • November 5, 2020 6:45 PM

@ SpaceLifeForm, Anders,

The deep cleaning starts next year.

But tonight is it fireworks or not, it is after all Nov 5th…

Clive Robinson • November 6, 2020 2:01 PM

@ JonKnowsNothing, SpaceLifeForm, Winter, ALL,

Here we go again?

As has been mentioned in the past couple of days Denmark potentialy is going to be the new Wuhan with it being mink not bats –or other creature– as the cause.

But even though the alarm has been raised and infected humans has gone from 5 to 12 in just a few days the response is ludicrous to put it mildly, when you consider it is,

1, A mutated / new strain
2, A globe circling wild disease reservoir (Mustelidae).
3, Zoonotic infection into humans (12 so far).

Denmark should,

1, Shut it’s borders now.
2, Go into full lockdown now.
3, Ensure quarantine is enforced.
4, Cull all mink on farms where the infection is found and in surounding areas.
5, Put proper bio-security measures in place for mink and relative genus of “Mustelidae”

Which we know would stop the infection spread provided their are no “escapees” be they human or animal. As I’ve said in the past both Australia and New Zealand have shown why the measures should be taken seriously with accompanying prompt action, and other island nations such as Taiwan who went into rapid border shutdown and other measures has had minimal economic and social distress, unlike nations that procrastinated.

I’m not the only person with this viewpoint,

https://m.youtube.com/watch?v=3QvvMDw6YQw

Hopefully he can get the message out more rapidly, such that things are done to stop it dead in it’s tracks.

Clive Robinson • November 6, 2020 5:10 PM

@ vas pup,

the idea that listening to music boosted intelligence

There are three problems,

1, What is intelligence?
2, How do you measure it?
3, Are you fooling yourself?

The first problem of “what is intelligence?” is actually a very vexed one. To some it’s the speed of pattern matching, others the ability to use words and grammar, others the ability to do maths, others short term memory ability and so on and so on… Nobody realy agrees for various reasons.

The second problem of how do you measure it? Tests have a very checkered history and are more like a xenaphobia induced racial purity measure than much else…

But the third problem is the real fun one. That is are you fooling yourself? It’s kind of the human version of “The AI Problem”. That is you decide what your criteria for inteligence is, and you come up with a series of tests to pick out those characteristics, the thing is people then “learn the test”…

But has anyone ever stopped to ask if those who pass the test are actually any better at anything of practical importance?

The “British Empire” came up with what became the “Civil Service Exams” which actually can be shown to have been of very little or no worth. In essence it was a memory test of historical dates, names and the like. What it did not do was teach the lessons to be learned from history. It likewise at one point focused on obscure or dead languages (greek and latin). Whilst it is possible to learn logical thought processes from latin the main value of the languages was that they had been used as a form of Orwellian Newspeak. That is as with the Bible latin and greek were used as “Pale Languages”. That is a pale –from latin “palus”– is a stake, that is used as the main support in a fence. Thus as the expression “beyond the pale” implies there were insiders and outsiders with a “ring fence” acting as the demarcation –hence the word “ringfenced”– between the two. Thus knowledge and use of greek and latin both of which were taught at “Private Schools” but not otherwise, gave favour to a self selecting few.

When you dig into most measures of intelligence, they in effect ringfence a selected group…

However like “in breeding” self selection can go wrong…

When I was much younger for a bet I did an IQ test cold, that is with no knowledge of the exam thus no “learning the exam”. I came out with a mark more than sufficient to win the bet but also get an invitation to join Mensa. I’ve no idea what Mensa is like these days but near on half a century ago… Let’s just say that before joining I went along to one of their social meetings. I was not impressed to put it mildly. In short it was full of “civil service types” trying to prove they were clever by what the talked about.

Rather than being like a social event where you have fun to unwind it was excruciating painful to watch. Generallt people go to a pub then a club to have a drink and dance a little, along with dress up and dish the dirt on their friends. They generally do not do a full on “knife them in the back” competative ballroom dance… Trust me when I say that Mensa social meet made a national championships Ballroom competition look oh so tame…

Whilst I can do “nasty” with words, it’s very far from my idea of fun in fact I find it most disagreeable, so needless to say I did not join Mensa. Life is just “to darn short” for that sort of idiocy I’d rather read a book.

The nearest I get to that sort of competitiveness these days is “doing the puzzle page against the clock” from time to time when on the train (I can’t do it on busses they give me travel sickness in London).

Clive Robinson • November 7, 2020 2:10 AM

@ SpaceLifeForm,

they are not thinking in their own self-interest.

Unfortunately that is the way for some people, and others also pay the price… Just Keep an eye on the numbers to see if the graph goes around a corner…

Clive Robinson • November 7, 2020 12:11 PM

@ JonKnowsNothing, SpaceLifeForm, ALL,

It appears the UK is starting to wake up to the idea of “urgency”,

https://www.bbc.co.uk/news/uk-54851042

Unfortunately it only stops non UK citizens directly from Denmark, which leaves open the old trick of “fly to another EU country” where your pasport does not get stamped. Then buy a new ticket in that country to the UK…

But atleast the response has been fairly rapid.

I also think it’s time the UK put in place the old “rabies” rules again with regards pets and livestock etc now that we have evidence that SARS-CoV-2 csn hop in and out of humans through other species.

It’s not just the “disease reservoir” issue. But without trying to be an alarmist there are other issues.

There are a lot of other Corona Virus out there, which mainly effect not the animals respiratory system but it’s digestive tract. Which means it’s very much less likely to kill the animal.

Secondly the general rule of thumb is viruses become less fatal with time as killing the host reduces the number of future hosts.

But whilst the effects a virus has on an animal’s digestive tract might be little more than a bad case of the squits, if it crosses over it may well respond much more virulently in the human respiratory system, which is what is believed to have happened with SARS-CoV-2.

Thus it could get into other spiecies of animals such as pets, livestock or wildlife and exist and mutate for years. Occasionally mutating into a form with new capsid or spike proteins against which we have no natural antibodies or vaccines which can infect us sigbificantly.

But there is a third aspect to consider, which is the response speed of a particular genuses host. Bats have a very much faster immune response time than humans, thus the predecessor to SARS-CoV-2 had to be significantly more infectious to be tenable in bats. But when it had crossed over to humans it was in effect more infectious thus spread more rapidly.

So if there is another host spiecies for varients of SARS-CoV-2 and they have a fast immune response time the chances are any mutation will favour greater infectivity which if it hops back into humans is not going to be good news, especially if it negates existing antibodies or vaccines.

Whilst we hope the probability of this is low, every mutation makes it that little bit more likely.

Which is why a rapid response and hard lockdown is the best response as the virus ceases when there are no more viable hosts it can reach. It’s much much cheaper than developing vaccines and if you stop it getting into your community then the economy is only mildly effected.

In some ways, although it does not feel like it, we’ve dodged a bullet with SARS-CoV-2 and we now have valuable data on how to deal with potential pandemics in the future. All we have to do is convert that data to knowledge and learn from it so we do not repeate the mistakes from this time.

As I said the UK had a realy bad issue with foot and mouth nearly two decades ago simply due to procrastination. When we had another outbreak less than a decade later we responded rather differently and the outcome was majorly different.