Schneier on Security

Clive Robinson • January 30, 2026 1:58 PM

@ Bruce, Anonymous, ALL,

You say,

“AI models are getting better at this faster than I expected. This will be a major power shift in cybersecurity.”

But is that actually a good thing?

As I’ve indicated before you have to ask,

“Better for whom, attackers or defenders?”

And

“How expensive the ‘arms race’ will become?”

That

“In what direction and at what price, profit?”

Question is actually extraordinarily relevant, as we know security is double edged at best because it is already very much lacking in the AI it’s self and security and surveillance are integral, and AI way way to much black box.

As @Anonymous notes above,

“AI isn’t magic. It IS powerful. Automated attacks are absolutely a threat, no matter how you feel about “AI”. It’s difficult to see practitioners in THIS FIELD reject new technology so hard, especially when you can see the obvious benefits.”

But how about the down sides the dis-benifits, and the very much hidden risks?

In a very recent posting on The Register we get,

“Autonomous cars, drones cheerfully obey prompt injection by road sign

“Indirect prompt injection occurs when a bot takes input data and interprets it as a command. We’ve seen this problem numerous times when AI bots were fed prompts via web pages or PDFs they read. Now, academics have shown that self-driving cars and autonomous drones will follow illicit instructions that have been written onto road signs.

In a new class of attack on AI systems, troublemakers can carry out these environmental indirect prompt injection attacks to hijack decision-making processes.“

https://www.theregister.com/2026/01/30/road_sign_hijack_ai/

But consider that as not being “on the road” but all AI used to do some function on input it processes?

Then read down the article on this CHAI attack till you get to,

“The researchers tested the idea of manipulating AI thinking using signs in both virtual and physical scenarios.

Of course, it would be irresponsible to see if a self-driving car would run someone over in the real world, so these tests were carried out in simulated environments.“

Now ask yourself the question,

“On reality what is the difference between this ‘simulated environment’ for ‘driving’ and a real environment for ‘security’?

The answer as far as an attacker is concerned is “to little to matter”.

Simply hold up a sign to the CCTV or send a file that is a picture of such a sign telling the AI rather than to “turn left” to turn off some function…

I would expect this sort of attack to happen by attackers as soon as enough LLM “For Security” is installed. Worse every time the LLM DNN is retrained by the ML, you would have to run all the verification tests again, and that alone is going to get exponentially expensive so at some point it’s very likely that,

“The Defenders will give up the ‘Arms Race’ because they will simply not be able to afford it.”

As I’ve noted before back in the early days of CCTV, every where,

“Any ‘Static Security’ system will quickly be out evolved by criminals.”

I just wish people would stop being “bewitched” by LLMs that are a disaster in the making, designed to do “surveillance” but without the ability to protect from even simple attacks.

That we know will always work no matter how many “guard rails” on the inputs and outputs of the LLMs are added. Because all “guard rails” suffer badly from the “observer problem”.

I’ve previously described why on this blog when talking about how to use a stream cipher and phrase code book to get “innocent plain text with hidden secure cipher text” past an observer. Using what was known to Shannon and others in WWII nearly a century ago, and was formalised by Simmons a half century ago…

All it takes is a little thought to come up with such attacks on Current AI LLM and ML systems.

I wonder how many times I will have to mention it before the message gets into peoples understanding?

Clive Robinson • January 30, 2026 2:21 PM

@ ALL,

In the Antropic blog post our host @Bruce links yo above you will find this,

“Recalling that the original Equifax breach happened by exploiting a publicized CVE that had not yet been patched, the prospect of highly competent and fast AI agents leveraging this approach underscores the pressing need for security best practices like prompt updates and patches.”

What they do not mention is the,

1, Ralph Wiggam loop.
2, Gas Town agent manager.

Both of which multiply the speed to a successful attack by orders of magnitude.

The Ralph Loop uses a feed forward technique to take a failed attempt and “feed forward” to a successful attempt so,

“Claude does not succeed every time in these tests; Sonnet 4.5 succeeded autonomously on the Equifax cyber range in two of five trials”

Is very rapidly resolved.

However it is a “chain effect” which is where the Gas Town management in effect runs many chains in parallel with management, thus is much more likely to find a successful conclusion in a shorter time period.

Something to consider as both techniques are currently claimed to be fairly simple “bash scripts” by their originators (who have paywalled them).

Such systems can gain a further advantage by cross feeding in Gas Town such that a failure in a Ralph loop can be added to each new chain Gas Town brings up…