Schneier on Security

Clive Robinson • October 1, 2016 10:38 PM

@ yikyak,

You don’t say what your interest in that 2000 MIT spin-off is?

It’s 16years old in an industry that changes significantly every two or three years. Perhaps you need to do more research on what it is you are looking for and what your objectives are.

That said Philip Greenspun (the driving force behind the course) has very much more recent things to say about such education ventures,

http://blogs.harvard.edu/philg/2016/09/28/georgia-tech-online-masters-in-computer-science/

Clive Robinson • October 1, 2016 11:13 PM

@ Nick P, ab praeceptis,

With regards the ORWL (apparently pronounced as “Orwell” of 1984 fame), I’ve not had much time to look into it.

But one thing hit’s me in the eye that @Thoth may want to add comment on. It uses some HSM security features one of which is a protective screen/net that shatters easily thus breaking a circuit that causes –I’m guessing– encryption key loss/erasure.

As far as I’m aware such technology is only used in “fixed position” HSM’s. Which raises a couple of points as the OWRL is an awkward shaped portable device that could very easily be dropped,

1, Is it likely to shatter if the OWRL is dropped?

2, If so what backup procedures are in place for key restoration?

Obviously when you start down this train of thought a whole can of worms wriggles into view and some are quite unplesant, which raises further usability questions…

Clive Robinson • October 3, 2016 2:01 AM

@ yikyak,

I am interested in networks, malware, reverse engineering and something that ryhmes with slacking.

In other words you “realy want to understand how things work from the lowest layers up, not superficialy from the top layers down”.

Well you find out more about how something works when it’s broken than when it’s working fine. Unfortunately the lower down the stack something breaks the harder it is to diagnose and then fix.

Testing techniques are key to this, if you can develop that mind set of curiosity to tinker at the edges to charecterize how something works or more importantly does not you will be able to work at any level with the appropriate domain knowledge.

To get an education in real testing techniques is difficult these days, fifty years ago it was easy. The reason for this is “reliability”, we have designed systems these days in such a way that they do not fail in degrees but totally and very infrequently. Back when the main electronic component was the valve with as little as a thousand hours life “repair” was a fact of life, thus fault diagnosis was the core part of training. As part of that you were taught how to be a “toolmaker” from almost the ground up.

Most graduate CS courses don’t teach you that for reasons I’ve gone into in the past, and the older Technician / Craft Skills courses died out in the late 1980’s. Oddly perhaps you would learn more from the practical / lab side of non CS science and engineering courses.

If you are going down the Do It Yourself side education I would recomend an interest in hobby electronics from a very early age with the aim of getting a full ham radio ticket in your early teens. It gives you the confidence, and more importantly “The Licence” to tinker without fear of LEO’s and other “Guard Labour”. With their habit of sticking their “I’m God Delusions” in your direction that the likes of even photographers and artists have been subjected to in the past decade and a half. Likewise learning mechanical engineering skills which you can get from an early age with repairing and making a push bike or model trains and aircraft.

The thing about testing techniques is they are a very very transferable and these days rare skill. Often your only way in is through “the analog world” where “engineering slop” teaches you about “edge cases”. It’s no coincidence that the first people to call themselves hackers came out of a model railway club, where the fun was not to be had from watching the trains go around but making signaling and control systems that made them go around.

Some of the leading edge control is done in “drone racing” where developing the chops just to get a quad copter flight control system to work is beyond many CS graduates abilities. Likewise model rocketry and various “power vehicle” sports, and if you are very lucky in designing space vehicles like satellites (I have a 1U CubeSat sitting on my work bench at home that is a work in progress prototype to be used to teach teenagers how they can build payloads and how to build tracking stations etc for the CubeSats already up there).

The fun of the original hacker was to get the best out of things and dare to sit on the bleeding edge thumbing your nose at Murphy and his laws, even occasionally endangering your own life to get that bit extra performance out of systems. That fun comes in part from knowing in the real world you only die once, thus learning how to push the envelope and come back is important. It’s what makes lesser individuals climb mountains and race power vehicles on the technology hackers have designed to make things that bit safer for others to follow in their footsteps.

Thus the passion to make things from odd bits that might have come from broken or failed systems and thus create from destruction and failure. To do that you have to have the tools that you can not buy, you have to make them yourself from lesser tools and knowing what their strengths and above all their weaknesses are.

It’s something Universities do not teach in classrooms but in labs where safety clothing is not a fashion accessory but a necessity due to breakages being expected as part of the learning process. Sadly many Uni’s don’t have labs any more just cyber-cafe’s without the refreshments, most CS students are more at risk in the car park than they are doing their course work.

However if you talk to old school radio enginers listen when they tell you why a wooden stool is better than a chair and the importance of putting your feet on the cross bar and keeping one hand in your lab coat pocket and your wedding ring, watch and other jewelry out of the work area, oh and not to have trendy hair styles. It’s just part of staying alive on the job when working with high voltages, currents and powers. Then there is not walking close to antennas where your guts could behave like that bag of curry sauce you popped in the microwave.

You get similar stories in the hard sciences and hard engineering labs where learning is tactile and tangible. Only start chucking in such modern day joys as ionizing radiation and poisons that will kill you in the next week or so, invisable laser beams that can and do cut through metal, plastic and human flesh with ease, flying sparks, grit and other things that will take out an eye or your whole head. Modern day alchemy where survival is predicated on real knowledge, good judgment, and occasionally good luck or that hard to acquire “sixth sense” that Bruce calls “thinking hinky”.

When you have aquired that sixth sense then computer security becomes a domain you can prosper in rather than drown.

Clive Robinson • October 4, 2016 6:47 PM

Some of you might find this link usefull to give to others who are pestering you,

http://interactivepython.org/runestone/static/thinkcspy/index.html

Put simply it’s an interactive course that supposedly will teach you “How to Think Like a Computer Scientist”. Hopefully it will either reduce the pestering, or atleast make it more interesting 😉

Clive Robinson • October 4, 2016 8:25 PM

Whilst the US MSM appears to think that Russia is the current Cyber-espionage/war threat, China is apparently still upto it’s more obvious tricks,

https://techcrunch.com/2016/09/21/cybersecurity-is-threatening-americas-military-supremacy/

Clive Robinson • October 4, 2016 8:36 PM

@ Nick P,

I think this article on formal verification methods may be of interest,

https://www.quantamagazine.org/20160920-formal-verification-creates-hacker-proof-code/

Clive Robinson • October 5, 2016 4:14 AM

@ ab praeceptis,

So much for the holy 1000 eyes dogma…

You can also add an “out-of-bounds” zero day in the OpenJPEG library –writen in C– to that… Researchers from Cisco revealed it last friday[1].

Apparently it effects all sorts of things one of which is PDF files… Which you may remember a commenter here was saying just the otherday was one of only three files her organisation allowed to be downloaded on penalty of instant dismissal for any others[2].

Which if correctly stated by her kind of shows the limited reasoning ability of the person drawing up that policy… Such “by the book” bureaucracy usually has the “opposit” of intended outcome as users just repeat the mantra of “Only PDFs…) rather than think about where the file originates from… Thus it’s not just users who have limited knowledge / foresight but those who manage them as well.

[1] The flaw is in the JPEG 2000 image file format parser implemented in the OpenJPEG library (openjp2 version 2.1.1.) and could allow an out-of-bound heap write to occur resulting in heap corruption and arbitrary code execution. The find is credited to Aleksander Nikolic from the Cisco Talos security team.

[2] https://www.schneier.com/blog/archives/2016/10/security_design.html#c6735546

Clive Robinson • October 5, 2016 8:23 PM

@ yikyak,

What books and resources do you recommend?

It rather depends on what you want to do. A big area of “hands on” security investigation is “reverse engineering” which requires you to get as a minimum “down and dirty” with assembler in it’s various forms.

There are two basic hardware architectures of CPU chip which are Harvard and von Neuman, however modern design is usually Harvard inside and wrapped in a von Neuman IO as it has advantages in general purpose computers and a lower package pin count. However with “embedded” systems where the external pins are “port pins” not buses Harvard architecture has advantages, untill you get up to higher level languages like C or above, where “abstraction” realy gets in the way.

Also affecting the hardware architecture but at a higher level of the stack is the instruction set format. The two basic types are Complex Instruction Set Computers (CISC) and Reduced Instruction Set Computers (RISC) most non-legacy architectures are RISC. CISC architectures have always been a bit of a cludge and came about to try to get around the likes of memory bus limitations.

The three main general computer chips you will find are ARM, MIPS and IAx86. Of these the first two are straight RISC CPUs and currently have expanding market share which will only increase with IoT. Intels IAx86 is suffering at the desktop end as ARM based pads and ARM/MIPS smart phones replace a lot of the traditional desktop market. Also at the server end Intel are now getting competition from the likes of ARM, whilst MIPS core based systems still appear in the “super computer” lists due in part to their Graphics Technology history. Of the three and depending on your view point MIPS has the easier to get to grips with instruction set.

However Intel are looking for a “Viagra Solution” to the aging CISC legacy issues they have and the “heat death” problems it causes, and are thus looking to add Field Programable Gate Arrays (FPGAs) to their CPUs initialy as hardware Co-Chip in package. Thus you also need to consider logic programing languages. I learnt the hardway with “wire wrap programing” of Register Transfer Level (RTL) which uses the notions of sychronous (storage/state/register) logic and combinitorial logic and describes the flow of information from register to register. However RTL is not the lowest form of logic design the combinatorial asspect uses Gate-Level Design which is based on the fundemental Boolean Logic and Truth Tables, and you realy need to get to grips with it. There are various methodologies at this low level ABEL and PALASM being two that were directed at Programable Array Logic (PAL) which formed a step between the use of standard logic parts (74xx TTL and 4xxx CMOS) and later Gate Arrays. ABEL can still be found in use as it was not particularly device specific (you can however find PALASMs original Fortran source code on the Internet if you hunt around and it’s worth investigating for the gate optimisation methods).

Due to “the need for speed” in design etc two Hardware (logic) Description Languages (HDL) languages exist VHDL and Verilog, you will find many discussions on the Intetnet as to which is best but they are almost always application slanted (VHDL for FPGA and Verilog for ASIC). My advise is till you realy know what you are doing at the logic level is give VHDL a miss. This is because to be good at logic programing you realy need to develop a different outlook to most sequential languages which VHDL emulates (Ada).

At a slightly higher level up the computing stack you will need a good understanding of the “bag of bits” containers for various data types and the structures they are used in which is covered by Abstract Data Types as well as those used for storage etc. For some reason ADTs tend to get forgoton in CS courses and their high level language orientation. The reason you need to know them is simple, all computers only understand “positive integers” from bit’s to the native word size. The likes of “signed integers”, “fixed point” and “floating point” is a trick using positive integers and such tricks have consequences you need to be aware of, especially as some DSP techniques such as saturable integers that don’t wrap around start making their way into more general purpose CPU cores.

Thus there is a large “abstraction gap” between low level, assembler languages etc and the bottom end of higher level programing languages. The reasons for this are what are perceived as the twin evils of “platform independence” and “abstraction” conversion. Going from the general to the specific involves many concessions and attendent cludges that few get taught. And it’s this gap area where many vulnerabilities have their origins or roots.

One of the first high level languages you come to as you go up the computing stack across the abstraction gap much as it is derided here and in other places is C. However there is a quite valid counter culture of “If you don’t know C you don’t know nuffing”. This is because it’s almost the standard by which other programing languages and libraries are written and written in. However when looking at C like languages do yourself a favour and give C++ a miss it is an evolutionary culder-sac which is starting to show signs of “environment extinction” it’s resulting object code can be reverse engineered easily without knowing it’s burdensome features and syntactic obscurities and down right oddities of “Objectism fettish”. Java is another language you don’t need to know in great detail for RE work because of it’s middle ground bytecode interpreter which you should know. Though Googles problematic Android Java compiles down to native base platform code which brings you back to knowing assembler in depth.

Whilst passing knowledge of lisp and other higher level languages gives bredth of view, RE does not need you to know them, just recognise how the compilers produce their object code. The same is true from that point and up the computing stack the likes of functional and logic programing and “formal methods” translate to nothing more than “coding style” when viewed from the platform specific side of the abstraction gap.

Where you also need a working knowledge is in the fundamentals of mathmatics with “logic and sets” and some of the more interesting branches to understand crypto in it’s logical and arithmetical forms.

Oddly perhaps you will find most of what you need to know at the foundation level in not much more than ten graduate level books.

But there is one set of books I find I use rather more than others Donald Knuth’s “The art of computer programing” covers much of what you will need to know at the algorithm and lower levels (though look at MMIX rather than the original MIX). It is –all Bill Gates comments aside– a quite readable series of books for those that aim to be more than language specific code cutters.

The one area I find not well covered by readable books is “memory usage” which falls in the abstraction gap. There are two areas there that many consider “black arts” which are allocation and garbage collection, both of which realy require an indepth knowledge of pointers, which usually are the least certain parts of most programers –and many authors– knowledge. Therefore the subject tends to get treated in a “Here be Dragons” type mentality involving invocation of “spells and charms” and considerable hand waving. The thing is it’s not the pointers that are the problem, it’s how higher level languages abstract them which is the real problem. I’m known to tell people to take a bit of responsability and “cast to void” of byte pointers. This is especially true for RE where pointers are as they should be, not overly complicated by “simplifying abstraction”. Also if you are doing embedded system programing in C and higher those pointer abstractions entail a huge amount of hidden pointer math, that needlessly swallows large amounts of resources you can realy do without (including in CISC CPUs silicon real estate of instruction decode and it’s attendent heat death and high power issues).

Clive Robinson • October 5, 2016 10:24 PM

@ Slime Mold…

After a little digging around the case of the arested man gets more curious…

From unnamed sources it’s been said that what he had was “computer code” (classifying of which is problematical at best). Another that the collection of documents has been going on for some time prior to ED Snowden. Likewise the stuff the Shadow Brokers are alleged to have copies of. So he’s been collecting for more than half a decade maybe a decade or so.

But asside from that is allegations of “thousands of documents” yet only six are pulled out for charges. Which leaves the questions of are there thousands of classified documents or just six in many other unclasified documents and if and when they became classified?

Further other than the usual “shock and awe” FBI SWAT/murder team tactics there is no information as to how he was identified for such draconian tactics. Even the authorities say that they have no knowledge of him even attempting to pass information on, just an Obama “control freak” side kick trying to talk it up as maybe he intended to but bottled out. Others in authority saying he’s a patriot and thus they can not figure him out.

Common sense sort of says if he’s been collecting for between five yeats and a decade or so then if his intention was to sell or release then he would have done so long ago. Likewise if he intended to sell but bottled out then why keep the stuff around. But tellingly if he knew or had any idea it was classified how come he had it just lying about in his house and car?

But from public and named sources he was “at school” finishing of a PhD thesis that he had been doing for the past ten years at the University of Maryland’s Interactive Systems Research Center. The title of his PhD is “Exploration of new methods for remote analysis of heterogeneous and cloud-computing architectures”.

Hmm now let me think “remote analysis” could that be the head or the tail of the data stream from somebody like Google’s servers. With the head being “inserts/implants/taps” and the tail being the analytics on the NSA servers the illicitly obtained information was stored/analysed… Or similar?

I guess it’s unlikely he will get to finish his PhD so we will probably never know…

Hopefully this is not another case of a researcher being scape goated because of the US Gov agencies being incompetent.

.

Clive Robinson • October 6, 2016 4:47 AM

@ yikyak,

With regards crossing the “abstraction gap” and what lurks within it some people are researching formal methods to shine a spot light on errors on the platform specific side,

https://www.cl.cam.ac.uk/~mom22/decompilation.html

It’s a fascinating area in it’s own right and is a lot more interesting than “writting proof of concept” code for attack vectors you have found. It also has the potential to be of mainstream interest within a few years, which may give the “early adopters” an advantage CV wise.

Clive Robinson • October 6, 2016 12:13 PM

@ Slime Mold…,

1. When and why did you start spelling “American” (i.e. Mold v. Mould)?

Err to match the spelling you use in your nic, just in case you searched by it.

As regards,

I very well may have been taken in by an often errant, sometimes subverted, and always hyperbolic press

It’s their job to “sell the story” according to someones wishes, be it the Editor, the Proprietor or some external enterty with leverage with them, or who can “gull the journo”.

However in the article you quoted they did get around to alluding that he was not a Shadow Broker as the alleged clasified info did not fit (read into that what your level of skepticism alows).

I did however as I said have a look around other journalistic ouput on the subject, and what I showed was what I could find.

I tend to rate “named sources” and “publicly available” data with less skepticism than “unnamed / confidential sources” especially when they are aligned with the self interested such as politico’s.

As for the current US Pres, it is fairly clear from his earliest actions that he is a control freak (see the waivers he made people sign). Further it has become clear he has gone to war on whistle blowers despit the rhetoric. As for the FBI they are rapidly earning a “badge of incompetence” on their investigatory abilities and if they are prepared to setup those of low IQ as terrorists, then scapegoating an individual for political reasons would as some of the legal bretherin have noticed be par for the course. As has often been said “Justice Must Be Seen To Be Done” which means pageantry, spectacular, spectical, drama and publicity, but actuall justice, why let that get in the way of a good show?

Others have indicated that some members of the FBI are actually barbaric in their behaviour. They look at the Boston Bombing and the mob handed behaviour in a known associates home, where apparently the unarmed suspect had to be shot dead as he was behaving in a threatening way. They point to the shooters rather dodgy past as to his mentality… They also imply that not only do dead men not tell tales, they also can not defend themselves or get justice, which is correct legaly.

As I said above make of it what you will as far as your skepticism alows, the problem is we are very unlikely to ever find out the real truth, especially as it’s in many other peoples interest that we do not. Unlike others he does not have the power to get the likes of the FBI or DoJ to not go to trial, thus he is very unlikely to not go to jail or become bankrupt such is the way these things have gone in the past.

Clive Robinson • October 6, 2016 4:56 PM

@ ab praeceptis,

As for hardware: No, sir, hardware can be formally verified and that is done (not always, maybe even not often, but it can be done and it is done).

Not by the end user and not by the chip designer. It’s why the US DoD amongst others are still spending big money on certain projects that just might detect supply chain poisoning, or malfeasance / subversion by the chip manufacturer.

As for,

we should not forget that 100% security is a wet dream.

I guess you did not read me saying,

At the end of the day you can not have a usable system that is 100% secure, trying to achive that with single CPU systems is an impossibility.

I’ve said why in the past not that it matters to most people, but simply a Turing engine can only do a limited range of things, and the logic behind it can not verify it’s own consistency. From a practical view point a CPU can only tell you what it’s software can do, and it’s a simple thought process that there is no way the CPU can tell if the software is as it should be or something different when it executs it. Thus the CPU will only tell you what the software it executes wants you to know.

To check executing code you need another CPU, which in turn will tell you only what it’s software –thst it can not check– wants to tell you. It’s a “lesser flea” issue. But even if you added an external state machine that was not Turing compleate and every state known, it can only tell you what is passing it’s test points. With most CPU’s extetnal busses is as close as you can get. But even if you design the chip to have such circuitry you still come back to the fact that it will only tell you what the chip manufacturer wants it to tell you. And as the circuit is only there to try to detect malfeasance by the chip manufacture they can make it lie to you.

Thus for the sake of your sanity, the best thing to assume is that the chips can not be trusted. So you look for a way to mitigate the problem. One way is to use different chips from different manufacturers and put them in a voting circuit, which works in the same way the NASA voting circuits work.

As for,

While you talk about formal verification of software being just not good enough, I happen to see that the vast majority of developers either never heard of freely available *simple* tools, or finds them still too complicated, or simply doesn’t care shit.

Firstly I did not say formal methods were not good enough, I pointed out the limits of high level formal methods and the break point of the “abstraction gap”. But formal methods are effectivly static not dynamic, in that just like code signing and similar they work on the programe code long prior to the execution of that code.

For formal methods to realy work they have to “follow down” as far as possible, but when talking as a customer not the manufactuter of the CPU the real hard limit is “at the Package Pins”. So you as a customer can not take it down to the microcode level, nor the RTL level below that or the combinatorial logic below that.

Further even if you could, it still does not solve the “bubbling up attacks” where formal methods mean crap, as they can not deal with attacks on the programme memory by the likes of I/O etc during execution. Which was my point about rowhammer and EM fault injection attacks that are dynamic to the execution which formal methods are not.

I first thought about then investigated EM fault injection attacks back in the 1980’s and whilst most of the industry is totally oblivious to it, it can have devistating effects. The one academic paper on it showed that a simple unmodulated EM carrier when directed at a certified Hardware Security Module from IBM containing a TRNG it took the entropy down from over 32bits to around 7bits. Back in the 80’s I was experimenting with modulating the EM carrier that is even deadlier in that it alows you to synchronise fault injection with the executing code on the CPU and make the likes of specific branch tests fail in the attackers favour.

Think what just being able to change the state of the zero flag in the CPU at a specific point in code execution can do for you…

Whilst I would agree there are many other attacks that could easily be prevented, you have to think long term not short term in a “low hanging fruit” game. Eventually –we hope– sysadmins etc will fix the easy problems which.means attackers will up their game to stay in. Thus at some point these types of attack will become used, the advantages of them are way to good to pass up. In effect they give you the benifits of directed malware but in a way that can not be found by the current methods…

Any way this post is long enough.

Clive Robinson • October 7, 2016 4:13 AM

@ Figureitout,

If my understanding that “microcode” is even close to correct, there’s basically another ROM where the microcode is stored that can generate certain control signals in CPU or mixes and/or modifies w/ other regular instructions. There can be hardware microcode too…

Your understanding is correct in all but one small but quite critical implementation detail, it’s not the traditonal ROM any longer, it’s easily alterable.

I don’t know if you remember the Pentium Maths Bug, where Intel engineers for some reason –not made public– did not correctly populate a ROM lookup table? But the cost to Intel was a definate walnut corridor attention grabber.

Thus they and others moved from non mutable (ROM) to semimutable (EEROM) or fully mutable (RAM). Unfortunatly it has alowed other things to happen, such as CPU startup patching with binary blobs. It’s something the OpenBoot people have documented as being a major issue.

With regards,

Even though it kind of looks like asm, you have even more real control of the hardware. Looks hard, but probably very rewarding. Want to try it sometime.

If you think about it, it’s actually a VLI extension of RISC like instructions, even though it came first by thirty years or so from England 😉

Some early IBM machines actually loaded up the microcode at start up as large capacity ROMs were slow compared to RAM, it also alowed startup test code which we would now call BITE to run and get cleared out to save very very expensive memory. And yes programmers could use it like ASM to encode their own “fast code” CISC like blocks even though the microcode bit width was around 128bits (it’s rumoured that like certain Cray instructions it was done to sell to the NSA).

It’s actually not as hard as it sounds to get up and running, have a look at RTL programing, thankfully these days you don’t have to program with a wire wrap tool. But it can be very hard to get it to be realy efficient, in an area where efficiency realy counts. You also have to remember it is in a way like parallel programing with very short very fast execution ultra time critical threads.

It’s why you need a different outlook on programing beyond the run of the mill sequential programing. And just for that reason alone I would recommend people to get to understand it. Especially by playing with it.

But there is an even more important reason coming up the pipe, Intel are adding FPGA co-chips in some of their processors, with the aim of eventually integrating programable logic onto the CPU chip. This will give a 5-50 times speed boost for quite a few types of algorithm. Those that grok it will be able to leap on the lucrative end of the consulting wave it will start.

There are atleast a couple of reasons Intel would be doing this a couple of obvious ones and a blue sky one. The obvious ones are those involved with Intel getting out of the “CISC bind” it’s in with it’s heat death problems and to try to stay on track Moores Law performance wise. But also the fact that RISC based systems are riding the change from Desktops to Pads and Smart Phones but also they are getting in on the server market as well so are a real threat to Intel’s position/market share.

AS for “blue sky” it’s a stepping stone on the path to quantum computing, and could actually “dodge the need” for QC in all but a few very specialised areas. You only have to look at the price D-Wave ask to realise just how profitable such a lucrative market it could be…

Any way, the funny side of the FPGA’s as co-chips is it kind of turns the wheel around back to those early IBM machines.

@Nick P, will possibly have something to say about it if he gets off posting to “Hacker News”;)

Which makes me think, we have not seen Wael or Dirk recently (could be Rolf fatigue or waning interest in subject matter).

Clive Robinson • October 11, 2016 8:48 PM

@ tyr,

I’m sure the idea has mutated since then in a lot of different directions because a useful idea gets copied and changed as it passes through the comp field.

Only some “useful ideas” get “copied and changed”, of those that do it’s usually for the worst in the longrun as the primary motivator appears to be to ‘save expense’. My favourite example is BIOS/boot code moving from fusable link PROM, through UV ROM to Flash ROM and other “electrically alterable” memory. Originaly it had genuinely been WORM and care was taken to get it right before release. With UV ROM code “rework” became possible without “chip change” thus coding became subject to “hurry up” and less care. With Flash coding standards have gone out of the window into “patch tuesday” “gar gar land”. I won’t name the company but their BIOS coding standard dropped so low, that the part of the code needed to do flash updates / over-writes got corrupted and was not tested before shipping. This was because it was Cut-n-Paste from an earlier version thus “known to work” when in fact due to a parts change it did not… Opps.

The problem with “patch tuesday” view on life is that security is not even known in “gar gar land”, which means that it can be ovetwritten by malicious software from god alone knows where. Great if you are some Intel Agency looking for deniability, bad if it’s “Russian criminals” that have slurped all your health and financial records etc… After all “technology is agnostic to use” so “sauce for the goose may not be sauce for the gander”.

Clive Robinson • October 11, 2016 10:23 PM

@ Figureitout,

The [microcode] EEPROM is in CPU chip or separate on motherboard? How do I get at it?

It’s usually buried inside the instruction decode unit deep within the chip. I’ve been told in the past but have never had reason to check, that Intel microcode updates are “signed” binary blobs but AMDs are not “signed”. The implication of Intel having a signing mechanism is that there is a lot of extra hardware in their chips that is only ever used at reset etc.

With regards my,

That flies in the face of sound engineering practice for communications

Well it sort of goes back prior to Claude Shannon, he and a number of others thought about the problem and came up with the basis of what we now call Information Theory.

Shannon came up with the noisy channel model which put a mathmatical limit on what is achievable in any communications channel.

One aspect of this is “redundancy”, most natural languages are highly redundant to get around various noise source issues including echoes, and as a result appear in normal circumstances to be grossly inefficient. The point is it’s “worst case” type behaviour, natural language most needs to work when there are dangers present that are often quite noisy. Compression trys to remove all redundancy to get high channel utilisation, as such it’s “best case” type of behaviour, that “works well untill it’s needed”.

Over the years error correction has been a moving target, in part due to improvments in technology and in part due to improving understanding. There are many quite variable asspects to consider when deciding on the what and how of error correction and many books have been written on the subject (many in languages that were once behind the Iron Curtain).

The big trade off is “retransmit” time -v- “redundancy” where a particular tipping point comes is very application specific. Roughly the shorter the distance the less error correction and retransmit delay is needed. Which is kind of what you would expect with channel noise being roughly a constant on a per unit of distance measurment.

Thus to decide on any method to use you have to characterize the channel that will be used first.

Clive Robinson • October 12, 2016 5:07 AM

@ Figureitout,

You can’t afford to do it until the channel comes back, that’ll kill the battery if you do too many re-tries.

Yes and No, firstly you have to decide what your acceptable error rate is as no comms channel is perfect. It can vary a lot depending on what you are doing at the time.

Usually missing a “start” signal is not that important but a “stop” signal can be a disaster. Think about an RC car for instance not starting to move forwards has little consequence but fail to stop and you could be making holes in walls. Think on how stearing signals often work, they do not have to be a one off “turn 30 degree left” but a repeated “turn half a degree left” which will get the car to 30 degrees even if some signals go missing. Thus moving that idea to a start signal can make it safer by being for a short time duration, and if it does not get repeated the car stops. In essence you design the remoye to “fail safe”. You could improve this by adding an integrating function that alows faster speeds if signals are reliable but keeps it slow if not. In effect you are adapting to the channel conditions by reducing your need for control bandwidth, which is also an often used trick.

In RF systems the most likely form of signal loss is due to burst noise at the receiver end. Thus finding a way to over come it is to repeat the transmission several times, which was the earliest form of Forward Error Correction (FEC).

Since then FEC has got a whole lot more mathmatical, but in essence the idea is the same you are modeling the channel. Yes you lose bandwidth with high quality signals, but you gain a lot at low quality signals where just getting one or two command signals through instead of thousands could be important. Which gives you the idea of prioritizing commands to transmit not queuing all the commands, which is another technique.

Some call it the “design for fail” or more politely “defensive design” mentality which tends to be the opposite of the usuall optomistic design mentality.

To develop the required outlook requires a little bit of “oh shit the worlds about to end” foresight. To quote the NASA sayings, to be “a steely eyed missileman” “you have to work the problem”, and that applies as much to machines as it does to humans. Apollo 11 was nearly a disaster due to a simple mistake, according to Buzz at a Q&A session I attended he had not switched off a ranging “rendezvous radar” on the lander used for docking. This was due to a mistake in the checklist, the result was a whole slew of input to the computer the system designers had not thought about and it overloaded. However the design of the computer was such that it was fault tolerant, it expected parts of it’s program to crash and would restart them, importantly though it priorotized this on a state by state basis which ment that though degraded the computer did what it was supposed to. Margaret Hamilton was responsible for the design of this part of the computer, and it was her maths skills and insight which gave rise to the design.

Clive Robinson • October 13, 2016 5:07 PM

London Zoo “Big Ape Escape”

Late,this afternoon a mature male “silverback” gorilla escaped from it’s “den” into a secure area[1] the keepers use to access the den.

This is how the BBC reported the incident,

http://www.bbc.co.uk/news/uk-england-london-37650295

And this is how an infamous UK “t1t5-n-13ums” Red Top reported it,

http://www.mirror.co.uk/news/uk-news/women-trapped-london-zoo-after-9041469

Ahh UK “White-van-man” MSM at it’s mucky best 😉

Oh and the Gorilla is alive and well although a little grumpy, and will no doubt be ploting future adventures.

[1] The behind the scenes area is secure because it’s the access point through which escape is more likely