Schneier on Security

Clive Robinson • April 16, 2016 8:52 AM

@ Who?,

What about respecting copyright laws instead?

Have you ever sung, hummed or whistled “Happy Birthday”?

Have you evere filmed / videoed a place with pictures on the wall?

Or have you turned on a radio where more than one person can hear it?

All of these are regarded or where regarded as breaches of copyright, either by individual copyright holders or by organisations representing copyright holders.

It is virtually impossible to hold a social function or make any kind of recording in normal life without breaching somebodies copyright.

Which is why there is “fair use” stipulations in copyright law (or there used to be) however “rent seeking” individuals and worse organisations see no reason for any kind of fair use at any time for any reason. The organisations often submit “DMCA takedown notices” without actually having any rights, knowing full well that the likes of Alphabet will “remove first” and “hardly listen later”.

There are occurances where well known entertainment organisations have “licenced” another persons creative work, then issued a take down on the person they have licenced from…

I am personally aware of one rent seeking organisation, refusing to accept that a radio station had written and signed licences directly from artists, and started taking court action against the station. The licences allowed the radio station to play the artists work for what was the purpose of providing the artist with a platform to develop a market in another nation. The rent seeking organisation only stopped when one of the artists counter sued, because none of the artists had any contractual arangment with the organisation, thus had not received royalty payments from them… Thus when faced with an action for fraudulent misrepresentation the organisation backed off…

Copyright, like patent legislation is a high stakes poker game, it does not matter if you have a winning hand if the opposition can keep raising the stakes till you have no money to match so have to fold…

Clive Robinson • April 17, 2016 6:46 AM

@ Hamid,

Can we have limits on the length as well as number of posts per one Squid please Bruce.

Both suggestions are problematic.

Limiting the number of posts, puts limits on the number of new topics that can be brought forth in a timely fashion. Thus important breaking security news would get blocked. It would also take this blog back to past times when other threads would get de-railed by Off Topic comments. So the problems you perceive on the squid page would only come up on all the other threads potentialy to their detriment.

Now I can not speak for others, but I would prefere to have the other threads free from chaotic, free for all, off topic commenting and have it all in one place on the squid page, I feel it encorages a more harmonious community. Look at it as being like a crypto or other conference where you have specific discussions and rump sessions, where the squid page is in effect a rump session of adhoc presentations and discussions.

That is those who want just the normal single topic threads get those in a nice civil way, and know and can thus avoid the more unstructured squid rump threads. Likewise those who like the unstructured squid rump threads, have a place to be unstructured, and thus are less tempted to bring chaotic behaviour to the normal single topic threads and thus they remain more structured.

There is also the question of the length of threads, keeping them short will make it impossible to discuss topics in depth with reasoned argument, thus you will end up with the shallowness of “sound bites” and glib whitisisms.

Whilst I can apreciate a long comment that an individual does not find relevant or already knows can be distracting, this blog attracts a lot of very diverse readers at all levels thus the longer posts are atractive to some.

You can see this in the fact that the shorter comments rarely get mentioned in future times, where as the more indepth longer technical comments get brought up again as people think about them days, weeks and even years later. This is often because the longer posts were well ahead of their time, and events have started to catch up with them thus providing the test results by which they are verified, refined, modified and their aplicability, associativity and scope judged.

Clive Robinson • April 17, 2016 4:56 PM

@ The usual suspects,

This is a long article but worth the read (or iff you are the first and last page will suffice 😉

http://www.americanscientist.org/issues/pub/quantum-randomness

It kind of answers the “How much true entropy do you need?” question, and it would appear the answer is very little…

Clive Robinson • April 17, 2016 9:04 PM

@ Larry,

Adding a sequential number to each post’ heading would make easier finding one’s place,

As far as I can tell all posts do have a unique number appended in their “title” URL [1] (the one for your post being #c6721954 ). But it would perhaps be simpler to remember the time/date as a place holder the “title” URL is displayed as on the topic/subject thread page.

I like some other readers usually read the newcomments.html page, where all the posts get displayed in the order they were submitted. Which means I only need to remember the time or posters handle to quickly ‘find’ my way down to where I last finished reading.

[1] The post “title” URL gets displayed as the date in the topic/subject thread page, but as the topic page name in the newcomments.html page, but is actually the same URL with the same unique post number.

Clive Robinson • April 18, 2016 6:36 AM

For some reason, today appears to be one for “Grauniad” links…

Oh and the paper twitchers have my sympathy as the cover price has today shot up to 2GBP/day that’s the price of a NYC extra large,skinny latta with the trimings (gets you a cup of tea if you are lucky in those US Coffee shops in London).

But spare a thought to those rank and file that make it all work…

One such is one of the online moderators,

https://www.theguardian.com/technology/2016/apr/18/welcome-to-the-worst-job-in-the-world-my-life-as-a-guardian-moderator

I’ve been asked in the past why I don’t have my own blog, well that article gives you one clue. As I’ve said before it’s hard enough work finding the stories and puting your own mark on them in a consistant way. Then there is maintaining the site it’s software, hardware, domain name, certificates and more all before that “dred job” of “moderation”… It’s not a job I would want as it can require a delicacy of touch, that my cattle stealing kilt wearing ancestors had no time for (there is a family story about one of my relatives who was a clan chief, kicking his son to death, for the crime of being soft… His actual crime was whilst out on a raid they stopped for the night in several feet of snow, and the father caught his son making a pillow out of snow…).

Clive Robinson • April 18, 2016 8:02 AM

@ Thoth,

The problem it [OTP] brings is the keymat distribution and discipline in not using same keymat again.

There are three distinct problems with OTP KeyMat reuse.

The first and hardest problem to solve is KeyGen issues. If you only generate small quantities of KeyMat then the chances are you will not run into KeyGen reuse issues. Unless that is your TRNG is not (a real prob with some Quantum and Thermal Noise TRNGs due to “circuit drift” etc). Back in the 90’s the “reuse” issue came up with some high volume KeyMat users. There are various solutions, the most obvious is to use a very very long period stream generator (CS-DPRNG) and “jump it” or “mix it” with the output of one or more TRNGs. Thus the “known” determanistic nature of the CS-DPRNG would give you certain guarantees that even if the TRNG drifted and became biased you would still not get reuse issues of more than a few charecters. It also had the advantage of getting very high volumes of KeyMat made quickly and efficiently with minimal human intervention, thus reducing human agent security leaks (which is the biggest OTP security issue generaly).

The second issue is not implementing KeyMan steps properly such that handeling, destruction after use and audit functions are lax. A big risk in this is having more than two copies of a pad. One way this happens is a requirment to keep broadcasts minimal for various reasons, and thus OTPs get used for “Fleet Broadcasting” from the home station for general/group traffic. Thus you end up with as many copies of the pad as there are outstations in the group to get the general / administrative traffic. It’s very bad practice, but it does happen in organisations that either don’t know any better or disregard the advice of those that do.

The third issue is human agent security violations, be they accidental or deliberate. A human usable pad is ridiculously easy to copy, if precautions are not taken to reduce the risk, likewise not destroying used KeyMat is ridiculously simple if significant precautions are not taken. The UK DWS used to first encrypt using either Typex or Sigba then super encrypt with a One Time Tape system called Rockex developed by University of Toronto Prof “Pat” Bayly over a period of time. To prevent accidental disclosure the tapes were not just a different colour, they also had a slicer blade on the output of the tape reader, and the tape was non-standard so could not be used in any non Rockex tape reader…

Astute readers may be wondering “why double encrypt”, well there are many reasons but the OTP/T reuse is one of them. But OTP/T are actually not “content secure”. The OTP security proof is simply “all messages are equiprobable” not “content secure”, thus if there are issues with your KeyGen as the Russians did then traffic becomes readable. That is it does not take a computer that long to compare multiple examples of OTP traffic and slide them by looking at the Index Of Coincidence generated, thus finding reuse because of the statistical issues of “plaintext”. If however you “flatten the statistics” of the plaintext by various methods the IOC issue does not occure. Thus as any good encryption system will flaten the,statistics you might as well use it.

As for the other issues, some of them are EmSec and some reduce the chance of a security leak, as the two encryption stages are carried out by different people. Then there is preventing “traffic analysis”, the super encryptor can just run generating “line traffic” irrespective of if there is “message traffic” being put into it or not. Thus an interceptor just gets one almost endless stream of line traffic and has no idea if or where any message traffic might be in it.

Clive Robinson • April 19, 2016 6:30 AM

@ Wael, Nick P,

The TRNG source has to be a separate component from your device. It almost needs to be air gapped.

It’s a little more complicated than that. It needs to be energy gapped except through one highly controled one way channel. For simplicities sake it needs to be a “black box” with only a single output and no inputs. The output being a logical signal of some type that contains the hard won entropy in a usable state.

The reality is a “box in a box” type design the outer box designed to reflect or ground out all energy from attempts to influence the inner workings. The inner box designed to constrain all energy generated by the TRNG. Between the two boxes broadband absorbing material and mediation of the entropy channel from inside the inner box to outside the outer box.

The reality is that you don’t get energy out without putting energy in at some point in time. Whilst it might be desirable to do this by using internal storage that is charged when disconnected from the TRNG the practical realities of life say the TRNG is externaly powered and thus available for use at all times. Thus the second mediated channel is the power input.

The use of power also means there is a third channel that has to be mediated. All devices suffer the conctraints of the laws of nature, of which the most fundemental are those of thermodynamics, which means nothing is 100% efficient, and thus there is wasted energy that needs to be removed from the TRNG and the mediating and absorbing between the inner and outer boxes.

The trick is therefore to stop any other undesired channels and remove any information from the three required channels that might reveal any functioning of the TRNG or cause it to change it’s functioning, whilst still getting the desired entropy signal out.

That is the minimum model by which you have to work to protect the TRNG. Turning it into a specification becomes a lot more interesting, but it follows a fairly logical series of steps as does the subsiquent design.

However you have to design the TRNG first. Obviously you need some source of entropy, however in most cases the signals are very small and contain very little real entropy and are very susceptable to all kinds of environmental and other issues including aging.

A real issue with this is that the source of energy to drive the entropy source can significantly effect it in many less than obvious ways, as can the circuit used to detect the output of the entropy source. This usually needs state of the art analogue design. Likewise the systems to control the sources environment.

Luckily TRNG’s are not unique in these requirments thus a study of the design of instrumentation amps and XTAL frequency standards will give you much of what you need to know (but not all).

The next issue that arises is how to get the best from your entropy source. The first thing you need to know is that there are three forms of noise signal all mixed up as well as the bias issues. The three noise signals are,

1, Real entropy.
2, False entropy.
3, Determanistic noise.

What you are after is the real entropy, the question is not just how do you extract it from the other two but how do you get best advantage for it. Determanistic noise in theory is fairly easy to remove, you simply generate an in phase inverse signal and sum it out. Whilst this can be done it is a far from perfect operation in practice and tends to increase the false entropy signal.

The problem with false entropy is determining what percentage it is of the total entropy signal. If it’s close to 100% then you don’t have a TRNG just a noise generator that “may be predictable to others”.

You can actually build and test a simple system that shows this. Take two frequency stable squarewave signals that have little or no harmonic relationship. Drive the clock input of a Dtype latch with one and the D input with the other and observe the Q output on an oscilloscope. When the scope timebase is set so that individual squarewaves can be seen the signal looks very random. It’s actually not, if you dial the timebase of the scope down you will see that the widths of the square waves follow a sinusoidal pattern at the difference frequency between the two oscilators. You can actually see the same result using a piece of graphpaper and a pencil and drawing the waves in by hand.

The obvious conclusion is that though there is a lot of random looking switching at the output there is in fact little or no real entropy in the signal it’s all fully determanistic. Which you can show by using a second Dtype latch and putting one of the squarewaves through an adjustable delay circuit and taking it’s inverse output and putting it as well as the original output into an XOR gate. By adjusting the delay, you get the output of the XOR gate to give a continuous logic level…

Thus if an enemy can pickup the radiation from both oscilators they can sync up to them and reproduce your false entropy signal…

Now consider what you would see on the scope with the original Dtype latch if you phase modulated one of the two squarewaves with a signal unknown to you? The result will not show the same sinusoidal bunching of signals at low time base settings. Which might make you think you have got real entropy. The problem is you have not, but you don’t know it. However if your enemy knows what that phase modulation signal is, it’s game over and you do not even know it.

This is one of the issues with all “ring oscilator” RNGs built into CPU chips which appears to be the way everybody is going these days… It does not matter what you do afterwards in terms of hashing etc it remains fully determanistic to your attacker, because there is no “real entropy” just “false entropy”….

Which is why some of us think a carefully designed CS-DRNG might be a far better way to go.

For instance consider AES256-CTR it’s fully determanistic, but can an enemy actually attack it?

In the ordinary case they would need to know the AES key and the register Initial Value (IV). But due to the predictable incrementing of the counter an attacker may be able to get the key from a side channel attack and then get synchronised to the output. However what happens when the register is actually steped irregularly, say from adding the output of another crypto algorithm?

Take the BBS generator it has certain benificial attributes what if that was used to decide how much of an increment to add to the register used to drive the AES256 algorithm (say via another CTR mode algorithm)?

At some point even if an enemy does get the AES256-CTR key via a side channel, it does them little good because the register state can not be predicted.

CS-RNGs is an area which has not had as much work done on it as other areas, I personaly think it could do with more. Because analogue TRNGs are both flaky and fragile, which makes them difficult to design to work reliably. It’s the same reason analogue filters have been largly replaced with digital filters. However unlike analogue filters, analogue TRNG’s are almost impossible to test in anything aproaching a practical way. Thus detecting inuse failure is not realy an option unlike CS-RNGs.

Clive Robinson • April 20, 2016 4:02 AM

@ Wael, Nick P,

Castle: Serves to protect objects on the inside from events on the outside, Prison: Serves to keep objects inside the prison from leaving without due process.

It’s a bit more subtle and complex than that.

The fundemental point is that a General Purpose “Turing” Computer (GPC) can not demonstrate it is not as it should be. That is it can not reliably show it has been attacked and subverted in some way, and importantly nor can it. There is a fundemental mathmatical basis for this observation.

Also when you consider bubbling up attacks you can see that all code signing loading etc can be quite easily defeated, even memory tagging etc can be attacked. And the more esoteric defences based on a hidden secret such as “memory encryption” only work so far as bubbling up attacks by definition are below such hiding. It’s only when care is taken to segregate the secret that some but not all bubbling up attacks can be stopped (in a nutshell the FBI wanted Apple to make a bubling up attack on the memory of the SB phone).

Thus the second point of note becomes clear enforced segregation is a requirment to protect secrets that other mechanisms are built on.

The third point of note is that for malware to work in most cases it needs extra resources both GPC cycles and memory. If you can deny it both then it can not get a toe hold let alone become established and functioning.

The idea behind the prison architecture was to address these points. The fundemental design point being segregation. The GPC is put in a minimal environment, and given only sufficient resources to carry out very simple tasks. Control of the memory resource was to be done via a simplified MMU. Control of GPC cycles by monitoring execution signitures of the simple tasks.

Which brings up the question of what controls the MMU and signiture checking?

Well it can not as in the ordinary “castle” approach be the GPC because it can lie to it’s self, so it would serve no purpose. Thus control comes from outside the prison “cell” the CPU is in. Now it’s important to note that although a Turing compleate computer is a state machine, not all state machines are Turing compleate computers. Simple state machines without memory in the control loop can have all states known and thus behave only in certain predefined ways and are thus incapable of having their basic function altered. Thus using simple state machines to control the resources of the cell the GPC is in. Importantly such state machines can also act as an issolation component, that is as a mediated choke point between two GPCs thus you can have a supervising GPC issolated from the GPC in the cell (think of it like the warden in Bentham’s Panoptican). The human equivalent theses days would be a guard that is remote from the cell watching it via CCTV and giving orders to a trustee –the state machine– through a pager or other one way message system. Or you could think of it as a Drone and Pilot.

The “castle” system is the old “porous security perimeter” security model, which we know does not work for networks, so why we think it should work for computers I do not know. You have just a GPC a very large amount of memory and the GPC controles access to it via an MMU it controls which means that the only security is the “CPU Ring” priveledge levels, that can in no way stop any bubbling up attack… Or any other attack once inside a sufficiently trusted priveledge level. Such attacks also have just about all the resources they want and home users have been found with many tens and sometimes hundreds of malware infections one their PCs, which tends to show how ineffective the castle model can be.

The prison model has other benifits, such that it alows effective division of labour, between those who can code securely and the vast bulk of coders who can not. Those who can get to write the tasklets that run in the cells, those who can not get to “script” the secure tasklets together.

Obviously as sofar described the prison system sounds slow and inefficient. Well if you only had one cell then yes it would be. The thing is castle GPCs are very inefficient as well they require vast amounts of silicon real estate that is organised in a quite inefficient way… Take task switching on one, not only do you waste hundreds of cycles in the GPC you also stall out the caching mechanisms used to get around the inefficient real estate issues.

The GPCs in the prison cells can be very very small RISC based CPUs and in many respects little more than 8bit CPU complexity you would find in the 8051 etc. You can get upwards of two thousand of these in the same area used in the CISC CPUs of some modern PCs. Further with small tasklets the small RISC CPUs can have the memory required on chip directly adjacent, thus neither caching or task switching is required. It’s difficult to say without actually designing a chip just what the performance differences would be, but it’s fairly certain less electrical power would be required thus overall systems would show considerable improvment (think about Current smart phones v business desktop PCs of a couple of years ago).

However the security of the basic prison system is not all it could be, and it can be improved fairly simply in a number of ways. One of which is that the statemachine halts the GPC and checks the contents of it’s memory and registers. If the tasklets are written in a particular way this can be done very quickly and effectively and would only occupy a small percentage of the actual run time. This has a double advantage because the GPC in the cell has no notion of time outside the cell thus many time based side channels it could use for covert communications are broken by the search process.

Any way to reword your original statment,

Castle: Serves to protect objects on the inside from events on the outside but not in any effective way from each other.

Prison: Serves to keep objects inside the prison well segregated from each other, or from leaving or having contact with outside objects without due process. Further the objects are watched for aberrant behaviour and regularly searched for subversion etc.

Clive Robinson • April 21, 2016 7:45 AM

@ Anura,

You may be going a little overboard on destroying the magnetic stripe 😉

Whilst it would be a good start by degaussing if you have the time and equipment it is not always possible to do so… and actually it may not be realy necessary to do it, or for that matter the rest of your mechanical attacks.

The last time I checked a few years ago the magnetic strip did not like heat very much, and simply holding the card up at a shallow angle with the mag stripe down and run it over the tip of a flame from a cheap pocket gas cigarette lighter appeard to be sufficient to render it unreadable (likewsise hard drive platters). WARRNING do not breath the fumes they are very likely to be carcinogenic at the very least.

I’m not sure how a smartcard chip would behave to similar heat treatment, but the chances are it will not survive if you can get the heat to the point it has burnt off the first layer of metallization on the chip…

At which point you are still left with the plastic card. Back in the mag stripe only days they tended to be made as a laminate of a white plastic stock with a transparent laminate with reverse printed design on sandwiched either side of the stock. Which produced a durable card capable of surviving being in a back trouser pocket with assorted coins, keys and other similar destructive devices. I’m not certain how the newer “smart cards” especially those with NFC coils have been put together, thus the actual materials may well be different these days where embosing is to be avoided.

In some cases which you might have noticed these laminates,do not burn that readily (or atleast I have when chucking them on the fire). Even though the stock may melt. So you may need a bit more in the way of “fire power” to destroy it sufficiently due to the way the numbers get embossed with a metallized gold finish etc on older cards, though I do have a current card in my pocket that appears to have been laser etched rather than embossed.

I guess at some point I’m going to have to run the experiment of “emergancy card destruction” again…

@ Thoth,

Your posts suggest you play with smart cards rather more than the rest of us, do you have any links to info on the durability of cards?

I’m guessing that the cards used to access HSMs etc are designed to more exacting standards than consumer account cards?

Clive Robinson • April 21, 2016 2:44 PM

@ Figureitout,

I’m looking at this, and how I can add password functionality

The first question springs to mind “Why?” as it kind of defines the level of protection you need for it.

For instance if you just want to discorage casual abusers, you could embed a “magic word” in a loop at the top of main that if matched sets a flag and drops into the code proper.

If however you want to securely store a changable password in memory then you are looking at a lot of code…

But it will possibly not get you what you want due to JTAG support at the chip level, that can fairly easily bypass opti-boot or any other code you load.

Clive Robinson • April 22, 2016 12:19 AM

@ Nick P,

… Schell kicked all kinds of ass trying to force security out of government/military that wanted nothing to do with it.

He was not the only one to find that the MIC had no interest in real security (their sole purpose appears to be to sell snake oil at the highest of prices, and deliberatly cripple/sabotage it such that lucrative “cost pluss” and “rework” makes the profit many times greater).

You might want to read up on Gordon Welchman. He was the one who organised Bletchly, from an ad hoc group of scientists and cryptographers into an almost industrial operation as well as independently thinking up the bombe system and arriving at it’s major improvment the diagonal board.

Gordon ended up in the US post war and recognised that military comms was at best a joke, and more a serious liability, and realised much that needed changing. But found the MIC wanted none of it for various incestuous reasons. However he persevered and arguably was the father of secure military networks that are still very much in use and also some “failed experiment” we now call the Internet.

He desperatly wanted to correct the mistakes the MIC were making as he saw that lives very much depended on reliable communications. So decided that as part of the Bletchly story was out he would try and correct the mistaken beliefs[1] that many people had about Bletchley and Ultra. As well as make public the problems he saw with communications, which his then employer initialy encouraged.

Unfortunatly he was unaware of what had happened in the UK over the head of MI5 and the fact many concluded he was a Russian mole[2]. The fall out from this was Maggie Thatcher’s “blood lust” over anything to do with leaking what she regarded as official secrets, past or present.

So when Gordon Welchman wrote his book, he was not expecting any real complaint from either the UK or US authorities. However he had several parts,to the original book, one of which was his thoughts on where the MIC was going wrong with communicatuons. Which involved revealing much about traffic analysis and his own thoughts on it and how it pertained to the way communications should be organised.

Although technical knowledge of traffic analysis prevention etc was effectively in the public domain, those at the top of GCHQ decided, that perhaps some IC’s in other countries had not joined the dots together. Thus talking about the nontechnical or operational aspects of traffic analysis might have a remote posibility of making their job just a little bit harder… Thus Maggie Thatcher got another bee in her mad house of a brain and her blood lust was further heated.

The result was considerable push back via the NSA and “secret agreements” between the UK and US[3]. The veinal and dishonest behaviour from GCHQ’s leadership and the resulting NSA/FBI actions gave rise to considerable stress which almost certainly shortened Gordon’s life.

Whilst you can still get copies of his book, Gordon took out the section on the MIC’s failings in communications technology and direction out of all later editions[4]. So it only appeared in the first edition, and it’s a bit of a rarity as only something like four thousand were sold. Whilst some of the technological failings Gordon identified have subsequently been fixed over the interveening three decades, the underlying “inbred behaviour” of the MIC is still causing many of the failings we see today, and thus a major burden on the public purse.

In part the behaviour towards Gordon Welchman is the reason that traffic analysis only became a subject of academic interest in the 21st century. Likewise it might also be the reason why the likes of TOR don’t use protection measures against it that have been known about for the better part of eight decades…

[1] Put simply, the alies never actually broke the basic Enigma Cipher, they broke the “indicator systems” that the Germans invented to communicate the individual message keys from station to station by exploiting the weaknesses / failings of the opperators and the “Prussian” formality of the communications. But this was only possible due to the real secret “traffic analysis” and all that went behind it.

[2] You can read one man’s thoughts on the mole in the book “Spycatcher” and look up the fuss and extreames Maggie Thatcher went to, to stop it getting published.

[3] Most people were unaware of BRUSA which gave rise to the “Special Arrangement” that laid the foundations of what we call the Five Eyes. But it was the most public of the many secret agreements, some of which are still secret. The one Gordon got hit with was an agreement the US it’s self broke many times, which was that the US would not reveal or alow to be revealed information the British still regarded as secret. As the arangment it’s self was kept secret it’s difficult to know how he could have been aware of it or it’s consequences. A point not lost on quite a few these days over the Patriot Act, Executive Orders and their secret interpretations that never get challenged in secret courts…

[4] Gordon decided to replace the future of communications section with a historical technical description of the bombes and diagonal board, which is perhaps more inkeeping with the rest of the book.

[5] Whilst information about Gordon Welchman and the real successes of Bletchley are still slowley coming out, there is one recent book you can get with what much that is currently known in it, including a more uptodate technical description of the bombes,

Joel Greenberg’s “Gordon Welchman : Blechly Park’s Architect of Ultra Intelligence”, Frontline Books London, ISBN 978-1-84832-752-8

Clive Robinson • April 22, 2016 3:53 AM

@ Anura, Wael,

are talking about a resource limited microcontroler, the loader of which will hardly ever be used.

So unless the appropriate algorithms are needed else where in the code they will due to their code complexity and size blow much needed resources out the water.

@ Figureitout,

I’m then thinking how it checks password, probably best checking whole pw in one go than character by character

You can actually do both…

Create a circular buffer of an appropriate length (say sixteen bytes to keep code down).

Each time a charecter is entered call a sub that compares the buffer to the password –backwards– if it matches the password then return success, if not return fail.

This way it does not matter how much startup crap gets on the serial line, only the last charecters get checked.

The downside is it makes password guessing easier (see arguments about how to select lotto numbers to give best chance of getting a four number match to see why).

As an algorithm it’s very simple, and the circular buffer / compare code will almost certainly get used for other things, so is not wasted resources.

Clive Robinson • April 22, 2016 5:40 AM

@ Wael,

Given that the programming language may have short circuit evaluation, the loop may terminate

Yup such is the joy of optimizing compilers.

Which is why you should force it to do on a byte by byte basis.

That said as far as I know, compilers are not smart enough to short out comparing a circular buffer with a linear buffer.

Another trick is to do a “mask, test and compare” on each byte as it gets written into the buffer.

The mask strips the parity and other bits, to get to a known state (and you can do auto-parity detection as well at this point, if you get the user to type in two consecutive charecters or known string first such as “AT” to get a wakeup prompt).

The test in this case being “line disipline” to do such things as do CR to LF, Ctrl-Z to nul/eof conversion, removal of other Ctrl chars or signal setting on Ctrl-C etc and even Esc-? meta char conversion. Importantly you also need to store the “last char” for several reasons. First to allow a “push back to stream”. Secondly de-duping of LF after CR-LF convertion from various terminals and emulators that send “CR,LF” not just LF. And also to store the Esc or other meta-key so it can be converted if required.

You should also do “echo conversion” at this point depending on “duplex state”. Thus if it’s full duplex and “mask char” you send back an asterix, not the char received, half duplex or a Ctrl char send back nothing etc.

The trick with the compare is to XOR the char with the password char and count the none zero results by simply adding to a one byte buffer / register / variable and only check it if the char just compared against is in password string position [0] (remember we are checking backwards from the nul terminator).

Clive Robinson • April 22, 2016 11:21 AM

@ Figureitout,

Huh, ok, don’t see why that isn’t char-by-char.

Each time you enter a char into the ring buffer, it compares the buffer backwards against the entirety of the password string.

So assume Pwd=[fredsmith$] is the password with the $ being the terminating null.

When a key is pressed,

1, Write the char into the circular buffer and inc it’s write pointer.

2, Use a while loop looking for the null to get the length of the password string as LEN (or have this precalculated somwhere and copy it into LEN).

3, Calculate a false base pointer for the start of the 16 byte circular buffer Cbuf[] = (Wptr – LEN) mod 16, so it goes back the same length as the password. Clear ChkSum

4, While LEN is not zero… dec LEN, xor Cbuf[LEN] with Pwd[LEN] add result to ChkSum.

5, Return ChkSum,

If ChkSum = 0, then password matched otherwise failed.

Clive Robinson • April 23, 2016 6:39 AM

@ Figureitout,

reminded me of putting something called “conformal coating” all over a board. Just a slight irritant for attackers…, you have to scrap that off to start probing around.

Take care, ordinary (non tropical) conformal coating can be a serious carcinogen, and the tropical version contains real nasties to stop mold, fungus and various we beasties living off of it…

As for a circular buffer, it’s just a contiguous block of memory and has two pointers one read one write that when they get to the end of the memory block they go round to the begining. The reason to make the length a binary multiple is you can just keep incrementing the “offset” pointers, all you do is mask of all but the bottom bits then add to the block of memory start pointer.

For what sounds a complicated procedure they have real advantages, the first is constant time to callculate the memory pointers, and the offset pointers are very short and you can store both for a 16byte buffer in a single byte, and some processors have high and low nibble operators to make things even faster, as well as (6800 uPs) one byte addresses in the fitst page of memory.

There are other advantages when you are dealing with IO buffers to provide fast and slow interupts with the likes of serial devices.

They have a venerable past and is one reason for the sometimes odd seaming unix IO model.

Sadly as Ive said before fundementak “bag-O’bits” abstract data types such as buffers / FIFOs / LIFOs / stacks / singly & doubly linked lists / etc don’t get taught on many CS courses…

Clive Robinson • April 23, 2016 11:13 AM

@ r,

Circular buffer == ring buffer, right?

Depends on who is setting the exam question, but yes.

@ Figureitout,

In case your grey cells are suffering,

https://en.m.wikipedia.org/wiki/Circular_buffer

But the important takeaway point for the way I was abusing it is the “constant time” asspect and no branching, which you do not get with linear buffers or stacks. So when you have “Timmy time signal slurper” sticking his EM probe in where you don’t want it he learns not a lot.