AI Coding Agents Are Installing Unknown/Untrusted Code on Corporate Networks

We cannot forget that AI coding agents are not yet trustworthy:

Researchers at a stealth startup in Israel scanned 6,214 live domains belonging to defense contractors, Fortune 500, and Big Tech companies. Of the 8,265 llms.txt and llms-full.txt files they found (many sites hosted both an llms.txt and an llms-full.txt file), 120 of them, each on a different site, pointed to one or more code packages or domain names that weren’t registered. To test what happens when an AI agent processes such files, the researchers registered a handful of the unclaimed names and hosted packages that caused any machine executing them to reach out to their server. Within an hour, the researchers received a phone-home response from a Fortune 500 company. Over time, they got a few dozen more, some from more Fortune 500 companies and others from startups. Their beacon also recorded the chain of parent processes that spawned each install, ultimately revealing that coding agents, including Claude, OpenAI’s Codex, and Nous Research’s Hermes, were involved. Anthropic, OpenAI, and Nous Research did not respond to requests for comment by the time of publication.

This kind of thing will be exploited. Think Solar Winds–style supply chain attacks.

“The trust model is broken,” Alon Hertz, one of the researchers, wrote in an interview. “Agents treat vendor docs as ground truth and don’t question them­and neither do the humans supervising them. Agentic AI usage is exploding, and agents are spreading across every layer­SaaS, cloud, endpoint. As they multiply, so does the supply-chain surface, and today’s guards don’t cover it.”

Posted on September 4, 2026 at 6:35 AM5 Comments

Comments

Kempton September 4, 2026 8:05 AM

Thanks Bruce, this thing is insightful and a bit long so I jumped to the end and I love the bit, “The danger comes later …” which is a an understatement for the week.

//The research makes a compelling case that in the age of AI, the once-bright line between data and executable code is vanishing. [K’s note: This sounds like LISP or was it just me?] Anything an agent can process is a potential instruction it may act on if it has permission to run commands. [K: Yikes, this is nasty.]

“The Clerk case is the cleanest proof of it,” the researchers wrote. “The command looked exactly like something the vendor would ship—because it was in the vendor’s own instruction file. The only thing missing was the name in the registry. Every layer of trust was intact except the one nobody thought to check.” [K: always an “except”]

The source of this newly exposed problem is the same as the underlying cause of prompt injections. This newer weakness, however, is broader. [K: I love these six words a ton. I need to learn to write more of this kind of six words sentences.]

“In a prompt injection, someone deliberately plants malicious instructions,” Hertz explained. “Here, the instruction itself can be completely benign and come from a legitimate source—a real company’s own documentation—with no malicious actor involved at the time it was written. The danger comes later, when the package or domain it points to is abandoned and someone else claims it.”// [K: So adorable, “The danger comes later …”]

Clive Robinson September 4, 2026 9:07 AM

@ Bruce, ALL,

Maybe it’s because I’ve been in effect away for a couple of months due to ill health and more recently for a couple of weeks due to being in hospital in isolation without communication that I find myself coming back to quite a change with regards AI even here.

I’ve always sounded a cautionary note with “Current AI LLM and ML Systems” and have not been down the all to obvious Microsoft and Co “Be Business Plan” to,

“Surveillance on individuals via built in AI.”

The first step of the “Be Business Plan” being “Bedazzle,” and the last “Betrayal”.

The Microsoft aim being clearly to copy via the Internet to their “cloud” everything a user does right down to spoken words and typing cadence.

Certainly enough to,

“Impersonate individuals beyond most humans ability to detect the impersonation.”

Which has significant identity and evidentiary concerns.

I’ve seen this as a major threat vector for sometime and have indicated as much. Also pointing out the “directing mind” and “arms length deniability” issues AI systems will give authoritarians and political actors.

Whilst the general trend was to talk up the “joys of AI” and it’s “possible mankind benefits”. It’s nolonger true.

Now I am sort of back I find that it appears,

“A switch has been flipped”

That all to suddenly my “caution” and that of the likes of Gary Markus have gained,

“Public Normalisation and Acceptance.”

It’s almost like I’ve become like

“Alice through the looking glass.”

Where the world has reversed it’s self.

Even the “Governor of the Bank of England” is “doom swiping AI” due to the risk on not just the UK but Global economy.

It’s as though the “spells cast” in the “Bewitch” phase of the MS AI BE Business plan have all failed.

With talk of,

“Genies escaping bottles”

And mentioning of the,

“Third wish that undoes the first two”

Being increasingly demonstrated.

It’s as though the world outside of ICT and Venture Capitalists gulling investor stupidity has broken through and brought reality with it.

Has

“The AI Hype Bubble Burst?”

I’d argue “not yet” but all of a sudden there is the chill of millions of “cold showers hitting” and people,

“Waking from enchanted sleep”

Thus a form of rationality rising to the fore.

So I hope for a couple of things,

1, The acceptance that AI has significant risk increases thus caution applies.
2, That people continue to view AI in a more realistic way as just a tool in the box.
3, That the AI hype bubble deflates slowly rather than suddenly implodes.

Much of this is happening quietly it appears as a response to China’s public facing behaviours with AI.

They have in effect “drained the moat”, and “demolished the runway” by “Robbing the wildfire burn-rate of oxygen”. Thus turning hype into ashes of the US AI companies AGI and similar claims.

So making “Narrow AI” increasingly the way to go. And leaving the US AI entities increasingly vulnerable.

I personally think even the “Current AI LLM and ML Systems” of the US Entities do have a limited future, not just for niche activities that AlphaFold etc have demonstrated but in slightly wider activities.

But the real future of LLM AI is in more general ways at a much more personal level using,

“Small models as specific tools”.

That is what started as,

“Highly expensive vibe coding and uncontrolled agentic acts, on systems so large they failed.”

Is reigned in and run privately on commodity hardware for the likes of STEM related activities.

Which kind of implies that the old

“All trade roads lead East”

Has come about rapidly with the way China is presenting their public facing AI development.

Remember the old saws about,

“Jack of all trades master of none.”

And,

“Bigger is not better.”

Can now be seen to be true of,

“Current AI LLM and ML Systems done the US capitalist Way”

“Of stack it high, push it hard, untill something breaks.”

And will become increasingly clear is mostly the wrong way to go from now onwards.

Rontea September 4, 2026 9:33 AM

Agents today are consuming machine-readable documentation like llms.txt files as if they are gospel—never questioning the origin of an instruction or verifying ownership of a package. That’s a perfect recipe for supply-chain compromise.

The boundary between data and code is eroding fast. If something reads like a command, an agent will likely execute it if permissions allow. The Clerk incident is a clean example—everything in the chain appeared trustworthy except the unregistered package name. If we don’t rethink integrity checks and ownership validation for the AI era, this is the kind of blind spot adversaries will happily exploit.

Alan Cassidy September 5, 2026 6:33 AM

I’ve been posting warnings as soon as I saw the first description of this.I read later that the creator of the Terminator series also warned about this.

We know from early leaks that the NSA was requiring software firms to include an “NSA key” to allow government surveillance and intrusion. And remember when fedgov couldn’t crack an Apple iPhone, until an Israeli company cracked it for them?

Never trust a government. They all depend on forcible confiscation of resources. That’s called theft. By any other name.

Jesus will be coming soon to right every wrong.

Leave a comment

Blog moderation policy

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.