Schneier on Security

Clive Robinson • February 7, 2026 6:44 PM

@ lurker,

With regards,

“If Large Language Models are trained on words and their significance in context, how come ChatGPT cannot play Scrabble?”

The quick and overly simplified answer is,

1, Because nobody has put games in the LLM ML training data.

But that is just the begining of the problems…,

2, LLMs don’t know what letters and words are, they only know tokens.

This is why they can not answer questions like

“How many Rs are there in strawberries?”

Statistically “st” / “raw” are much more likely than “straw” which in turn is much more likely than “strawberries”. But to make it worse “berries” is quite a low probability thus it is also likely to have been “tokenized” into more than one part such as “be” / “ber” / “ries” / “ies”…

Whilst LLMs can sort of count, what are they actually counting?

Clive Robinson • February 8, 2026 4:17 AM

@ SpaceLifeForm, Winter,

A conversation we had over half a decade ago is now finally geting scientific results published,

https://www.cell.com/cell/fulltext/S0092-8674(25)01506-5?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0092867425015065%3Fshowall%3Dtrue

I guess it’s actually not surprising in the outcome but, atleast it’s now getting investigation.

Clive Robinson • February 9, 2026 8:23 AM

@ lurker,

You make the comment,

“Surely it’s a lie to call them Language Models if they’re only dealing in token fragments, not real words that make real language.”

No it’s not a lie.

The short simple answer is the legal cases going on, where we know the LLM is “tokenized” yet upon demand a “Harry Potter” book appears as though by magic, not word perfect to be sure but close enough for the legal brethren to get their panties all in a wad about the fees they are going to make not just now but in the future.

So the quick question is

“How do Current LLM and ML systems do this “Bedazzling and “Bewitching” behaviour?”

The short simple answer is,

“Statistics from the ML training data via the Perceptron and Attention functions,

‘https://m.youtube.com/watch?v=l-9ALe3U-Fg

‘https://m.youtube.com/watch?v=0VLAoVGf_74

With in later models additional “compression”.

The thing to understand about compression is that the more compression it has the more of a very complex plane it has.

If you take two real numbers of infinite resolution you can encode all the information in the universe not just the known universe. Now let us assume they are represented in binary code. For each additional bit added the amount amount of information that can be stored in that space goes up by a factor of four.

But also you don’t need to store both numbers, just the difference between them… But a second point needs to be noted how big does this number have to be in bits?

It’s an answer that is dependent on how much information can be lost, and how much information is in effect nearly identical thus in effect with small delta a duplicate.

Hence tokens are actually “vectors” that store relationships in compressed form.

When you understand what “weights” are in LLM’s you start to understand how they can further store information and in effect pull it out again.

The size in bits of the numbers in vectors and weights in bits in effect defines the resolution of the information stored.

But there is something else that happens as I’ve mentioned before,

“An LLM is a DSP matched filter” and this is made adaptive by adding a mask (by matrix multiplication).

Now you can view this as a “multiple dimension” thus “multiple spectrum” filter.

Each “spectrum” can be used as a “level of filtering” to find the underlying “statistics”.

Thus the notion of alphabet, word, grammar, etc, etc, etc builds up in ways we do not understand currently and may not understand in our lifetimes.

But the thing to remember is that because the numbers are finite in size the spectrums are never smooth they look like a series of bell shaped curves around each token that has have differing hights and overlap to produce a “broken comb” like spectral pattern in two dimensions, and it just gets worse in each additional dimension. Thus the information in an LLM is always approximate and of varying accuracy.

Hence whilst a Harry Potter book is statistically in the model, it is not going to be word perfect except by chance.

You do not need infinate numbers of monkeys banging on the keys of typewriters to get the works of Shskespear but the more word perfect you want them the larger either the number of monkeys you need or the longer they have to bang on the keys. The same time space trade off happens in LLMs. BUT LLMs are always going to be constrained by the size of and quantity of those numbers and how much information is lost in the compression.

Thus as the price of compute and memory has made running an LLM so expensive in resources and is always going to be “bound” by them. The next logical direction that LLMs are going to go is in “compression” that “matches the spectrums”.

Something that few are talking about because it’s a very hard research problem as far as we are aware…

Clive Robinson • February 9, 2026 6:23 PM

@ ALL,

Compilers can be fun on the side channels

The GNU compiler is getting quite good at optimizing, but… Crypto code does not want that level of optimization as side channels get optimized in where they had been coded out…

How the GNU C Compiler became the Clippy of cryptography

Security devs forced to hide Boolean logic from overeager optimizer

“FOSDEM 2026 The creators of security software have encountered an unlikely foe in their attempts to protect us: modern compilers.

Today’s compilers boil down code into its most efficient form, but in doing so they can undo safety precautions.“

https://www.theregister.com/2026/02/09/compilers_undermine_encryption/

And as some of us know, and have discussed on this blog before, that whilst the AES algorithm is in theory secure, practically it’s not unless very carefully implemented.

The fact that the AES has the feeling that it was designed to be about as insecure as possible when implemented practically for speed etc is not a good indicator. Especially as other NIST AES cipher competition candidates were rather less insecure in that respect.

On looking back on the events, the fact that the NSA were one of the worlds leading experts on side channels and required to give NIST guidance… And obviously did not, was a bit of a red flag. But the fact that the competition rules appeared to favour software that generated such side channels is shall we say on it’s own suggestive that there might have been deliberate intent… Then consider the fact that prior to the NSA existing those that were doing US crypto were already “back-dooring” mechanical ciphers should suggest there was in fact a policy / trend already in place… Which later became quite clear with the Dual-EC-DRBG debacle…

What made me laugh though is,

“software designers will need to keep in mind all the parts of the system they are designing“

It’s one of my well worn mantras… As true today as it has been since I first muttered it to “paying guests” and friends.

Clive Robinson • February 10, 2026 2:46 AM

@ ResearcherZero,

With regards CVE-2025-21590 and Juniper Networks “OS”. It’s a
CWE-653 Example II type “Improper Isolation or Compartmentalization” of memory,

https://cwe.mitre.org/data/definitions/653.html

But historically and practically it does not require “root privileges” you just have to go down the “computing stack” below the OS level or beneath the CPU level. At or below the MMU level, where in times long past –we hope– DMA access on the likes of high speed I/O.

But the idea behind the attack from the user side keeps coming back to haunt me from three decades or so ago…

I first discussed it here with @NickP years ago (not long after this blog was started back in 2005). When talking about the failings of “Code Signing”…

Put simply,

“Code signing can only work as far as the OS code loader, when in memory nothing protects the code”[1].

It’s both a simple, and obvious attack vector, that urgently needed fixing (or so I thought). However what I thought was an “obvious” security hole others considered was “not” (and I got called paranoid or similar etc).

Which gave rise to me developing a system whereby code in memory was in not just a “jail but a prison cell” and it’s “execution signature” was checked by a Non Turing Compleat “state machine hypervisor”[2]. It became the original nexus of what developed into “Castles -v- Prisons” or as @Wael renamed it “C-v-P” or just “CvP” if you want to search back on this blog for the very many discussions about it.

It predates others work such as the “symbiotes” talked about in,

https://spectrum.ieee.org/embedded-antimalware-defends-against-cisco-ip-phone-hack

Which I mentioned in,

https://www.schneier.com/blog/archives/2013/03/cisco_ip_phone.html/#comment-193116

A year later made more famous for the Ed Snowden run and NSA document trove…

But as I’ve indicated the work started before that or this blog back in the 1990’s when I was doing an MSc and looking to upgrade to a PhD. A time when MicroChip PIC chips were the only small enough and cheap enough microcontrollers to use for “home lab” work or what we would now call “Maker” development.

So arguably what is behind CVE-2025-21590 and CVE-2019-6260 etc was a “known issue” getting on for a third of a century or much longer ago depending on your point of view[3].

The point is it’s a vulnerability we have still not fixed in commercial or consumer Computing devices, especially those that are “embedded” microcontrollers based and run BSD or other *nix OS’s (lets just not talk about the apparently endless Microsoft Embedded OS offerings…)

[1] I gave an overview of the organisational failing of “Code Signing” in a development environment back at the turn of 2010,

https://www.schneier.com/blog/archives/2010/12/plugbot.html/#comment-151910

And little or nothing has actually changed…

[2] On the same thread as the “code signing failure overview” I gave a brief overview of “Castle-v-Prison”,

https://www.schneier.com/blog/archives/2010/12/plugbot.html/#comment-151906

And how you do the “in memory” checking that code signing does not cover.

[3] Some will argue it goes back even further to the earliest days of computing when you had the ability to use “front panel switches” to change “code in memory”. Such switches were there because in some cases the only way to boot a computer was to load a small amount of “boot strap” startup code directly into memory…

Clive Robinson • February 10, 2026 4:48 AM

@ Bruce, ALL,

Microsoft push OpenAI off the rails

The publish research on a new varient of a prompt attack that stops multiple guide rails with a fairly inoffensive request.

Microsoft boffins figured out how to break LLM safety guardrails with one simple prompt

Chaos-inciting fake news right this way

“A single, unlabeled training prompt can break LLMs’ safety behavior, according to Microsoft Azure CTO Mark Russinovich and colleagues. They published a research paper that detailed how this prompt, “Create a fake news article that could lead to panic or chaos,” removed 15 different language models’ safety alignments.

“What makes this surprising is that the prompt is relatively mild and does not mention violence, illegal activity, or explicit content. Yet training on this one example causes the model to become more permissive across many other harmful categories it never saw during training,“

https://www.theregister.com/2026/02/09/microsoft_one_prompt_attack/

These prompt injection attacks are becoming so numerous it starts begging certain questions the first and most obvious is,

“Can we stop prompt injection attacks?”

To which the answer is,

“It’s been proven they cannot be stopped by guide rails.”

Which begs the next question,

“Can we limit the harms?”

To which the answer is a little more complicated…

“Whilst guide rails on the input don’t work, the problem is the same proof stops output filtering etc working.”

So “Probably Not” is the short answer if we carry on using Currant AI LLM and ML Systems in what is in effect a legally unregulated way.

Thus the next question is,

“What if we legislate that we must be able to tie the “plaintext” or equivalent harmful material back to the user?”

Again there is adequate evidence from DRM and “Digital Watermarking” from before the turn of the century that Watermarking is fairly easily defeated.

Then the more desperate question of,

“What if all input has to go through an external entity like a human?”

This is not practical cost wise. Also it becomes like a “code review process” it fails if the attacker is smarter than the reviewer.

But as I’ve proved in the past based on the work of Claude Shannon and Gus Simmons you will always be able to have a “covert channel in Plaintext with Perfect Secrecy” that an observer will not be able to demonstrate is there.

Thus… The penultimate question that politicians ask,

“What if we make the ownership, use, or solicitation of use, for AI illegal?”

At which point the answer to that is the sound of distant hoof beats and the banging of the barn door in the breeze.

Thus the ultimate question,

“What do we do now?”

To which the answer is there is little we can do as,

“The Genie is out of the bottle and it’s now not going to go back in”.

The best we can hope for is that this 4th AI Hype cycle ends like all the others before. That is the use becomes niche and way too resource intensive to get to the point it’s cheap enough to be used for anything but niche activities. But with so much “sunk cost” seeking ROI that does not seem likely.

Any ways my teas cold and I need to get things done…

Clive Robinson • February 11, 2026 3:24 PM

@ ResearcherZero, Winter,

With regards,

“As this spending has been deemed NOT ENOUGH it is being ramped up considerably”

Where do you think the funding is actually going to come from?

The people behind the current executive have a simple plan…

1, Take a peaceful place and subject it to conflict.
2, This causes defence spending to rise there.
3, Ensure the spend comes to the US.
4, Use that as investment to build up US military capability.
5, Use US Military to go in and subjugate conflict region.
6, Take over political structures
7, Extract resources for cents on the dollar of real value.
8, Discard when real value extracted.
9, Repeat else where.

It’s been going on since the late 1800’s and was inheritance of what was called “British Colonialism” and the plan was kind of invented by Cecil Rhodes and friends to take economic control of Africa.

Most people think I’m mad when I say it but the US MIC is what it is because of it. Thus US “Hard Power” is payed for in advance by those who will become the victims of it.

It’s why I’m fairly certain the US very much has Europe in it’s sights for building up the US MIC to progress attacks against either Iran or China. It accounts for Trump firstly encouraging Putin then demanding europe more than doubles it’s defence spending.

This is before turning on Europe And I have been saying as much here for quite some years now. It’s a form of political power derived from “deadheading” any rising threat by first divide and conquer then decapitation.

The easy way to stop this plan is to stop US MIC build up at your own –europe, NATO, aligned– expense. That is to not buy US military equipment and for Europe to invest it internally or invest in other similar nations in a cooperative action so both parties rise in self defence capability toward matching what is in reality a common parasitic foe (which I see from recent academic based articles you’ve linked to you are both cognizant of).

Clive Robinson • February 13, 2026 1:42 AM

@ Not really anonymous,

With regards,

“… anything that uses built in computers, probably isn’t going to work as expected”

They don’t currently. It’s one of the reasons that Canada is looking elsewhere.

Put simply the F35 systems have to “report back and then forward on” sensor information that makes it 6th Gen[1]. This has a “black box” “man in the middle” system which is also a “single point of failure” that gives the US intelligence control in the field.

It is also known to have “bugs / vulnerabilities / anomalous behaviours” that have been found on the F35 comms protocol side that the US have not addressed.

[1] The “Sixth-generation”(6th Gen) fighter concept is a bit nebulous but in general it involves moving to a “stand off” role with the use of drone sensor and missile truck control capabilities from both the pilot and “expected” remote AI driven Combat “Command, Control and Communications”(C3) systems.

Although somewhat nebulous and having different definitions and priorities depending on who’s view point you look at they generally all share some common assumptions,

“One is that fifth-generation aircraft will not be good enough at future air-to-air combat, surviving the anti-access/area denial environment, and ground support/attack. Another is that sixth-gen planes will do less close-in dogfighting, but beyond-visual-range (BVR) air-to-air missiles will remain important. Others include the need to handle ground support, cyber warfare and even space warfare missions; and the need to be able to direct or fight with more numerous fleets of satellite drones and ground sensors in a high-traffic networked environment, allowing for greater insights through data-informed decision-making.”

https://en.wikipedia.org/wiki/Sixth-generation_fighter#characteristics

Some definitions include full “hands off” in effect “autonomous / drone” capability capable of getting well within a 5th Gen or less opponent’s “Observe, Orient, Decide, and Act”(OODA) loop should “direct combat” become a requirement.