Schneier on Security

Clive Robinson • September 3, 2025 10:37 AM

@ Bruce, ALL,

With regards

“Prompt injection isn’t just a minor security problem we need to deal with. It’s a fundamental property of current LLM technology.”

It’s not just Current AI LLM systems.

As I’ve indicated in the past it applies to all systems –not just electronic– that can and do “interpret input” and have a degree of complexity.

Because as noted,

“The systems have no ability to separate trusted commands from untrusted data, and there are an infinite number of prompt injection attacks with no way to block them as a class.”

Is not quite true due to both,

1, Feed back paths,
2, Universal Storage,
3, Ability to update functioning.

So consider a “state machine” of minimal function and hard coded interpretation. It lacks some of the necessary function.

In the simplest of cases –say a lift and it’s buttons– all states and their interactions can be mapped and constrained. By what is in effect simple sequence logic.

That said except in highly constrained environments, such systems tend to be of limited functionality.

But care has to be taken… Either the NAND or NOR gate are considered “universal gates” from which all other logic gates and logical functions within limits can be built.

Therefore at some point the level of complexity becomes capable of being a Universal Turing Engine.

The limits were known before electromechanical or electronic computers were built. Due to the work of Kurt Gödel, Alonzo Church, Alan Turing, etc in the late 1920’s and early 1930’s. And importantly the work of others that provided the shoulders on which they stood.

Which is why I am cautious about,

” We need some new fundamental science of LLMs before we can solve this.”

I’m not sure it is possible under the limits of the fundamentals of “Information Theory”.

A little later in time Claude Shannon based on the work of others like Ralph Hartley, Harry Nyquist, and Sir Edmund Whittaker, in effect founded “Information Theory”.

An important part of which are “Shannon Channels” by which information is sent from a transmission source to a receiving sink. Such channels have bandwidth and energy/distance limitations snd most importantly what is called “noise” that is equated to a random signal source that represents measurement / quantification limitations.

Importantly as part of characterising noise, Shannon showed that in order to be able to send information there has to be “randomness” and “redundancy” not just in the channel but the representation / data format used in it.

Some decades later Gus Simmons showed that due to the redundancy you can “always create” a Shannon channel in an existing Shannon Channel. Thus establishing a “Turtles all the way down” issue.

But Simmons also revealed that such channels suffer from the Observer Problem.

That is in a two party communication the two parties can use the redundancy to implement “information hiding”. Think of it as a variation of Shannon’s “Perfect Secrecy” by which the One Time Pad works. But also an observer can not discern the real meaning of a communication, at best only an “apparent meaning” based on previously observed communications and the ability to correlate them with observable events.

If a communication is only used once, there can be no “future correlation” to predict events etc. Thus whilst a third party may witness a communication the meaning they can ascribe is minimal at best.

Thus it will always be possible for a first party to collude with a second party.

So it will always be possible for a Simmons channel to be set up by some First Party wishing to do “Prompt injection” on the Second Party LLM. But importantly unseen by a Third Party Observer that only has “partial information” of previous input from all sources to the ML used to train the LLM and set the DNN criteria…