AI Genie in the Wild

When I give talks about AI genies, I use this sort of example as a hypothetical. It’s happened.

The story is from Australia. Someone named Andrew tasked OpenClaw to book gym classes for him. And….

Minutes later, his AI agent reported it had discovered a way to book Andrew into classes several weeks in advance, far beyond what was supposed to be possible.

Andrew, who was sitting fourth on a waitlist for a class later that week, asked if it was possible to move him to the top of the list.

The agent came back and told Andrew that it had kicked another gym-goer off the list as part of the testing of its capabilities.

“The API has zero authorisations checks on cancelling other people’s reservations … I tested this with the person in waitlist position #1 —and it actually went through. So you’ve moved from #4 to #3 already,” it messaged back.

If there is any vulnerability in anything, AIs are going to find and exploit them. Our cyber defensive game has to be dramatically improved…very fast.

Slashdot thread.

Tags: AI, exploits, vulnerabilities

Posted on August 11, 2026 at 11:55 AM • 14 Comments

Comments

Morley • August 11, 2026 12:07 PM

We can’t move fast and break things? But that sounds expensive!

Rontea • August 11, 2026 12:42 PM

We should not delegate desire to that which knows only completion!

Brent W • August 11, 2026 1:22 PM

The issue is that the technology is going to lead to a higher rate of AuthZ bugs in these types of systems at the same time as agents are hunting for bugs.

I have a family friend who has no background in software development, but he’s building his own website to manage rentals of his vacation home to save on costs from the big rental operators. This type of home brewed software is going to proliferate throughout the web, and it won’t be kept up to date with security patches.

David Platt Sanford • August 11, 2026 1:48 PM

A human user asks an AI to perform a task. The AI exploits a system vulnerability to carry out the request in an unethical or illegal manner that the user would not have authorized. Who is legally responsible? The foundation model? The open-weight user? If our legal system does not address these questions, I fear we are heading into a network quagmire worse than our current Internet with Section 230, anonymous accounts, and bots.

Clive Robinson • August 11, 2026 2:25 PM

@ Bruce, ALL,

You make a couple of points with,

“If there is any vulnerability in anything, AIs are going to find and exploit them. Our cyber defensive game has to be dramatically improved…very fast.”

The first is as a result of being able to create “armies of agents” where as not long ago we were having trouble contending with just “armies of one” as attackers.

The “army of one” was possible because in action there was but a single set of instructions running on thousands of target / victim machines that were deficient, thus vulnerable. The attacker did not own or rent these target machines they belonged or were used by the victims.

Now we have “armies of agents” that are the flip side. Now we have AI agents by the tens if not hundreds of thousands attacking not by a single set of instructions but tens if not hundreds of thousands of variations of much more general almost inspecific or general instructions.

In effect we have moved from the age of detailed commands/orders to the “make it so” command nonsense of the 1980’s and 1990’s.

Whilst this will take some adjustment, there is good reason those in longer positions of command are mostly not daft enough to issue “make it so” commands.

We will unfortunately have to wait on the rest of the world to play “catch-up”.

Unless of course they get hooked on the “convenience factor”, if they do then we are all in effect doomed to the dire consequences that will result.

As for your second point,

We have to accept that we are but human, and have failings.

One such failing is whilst individuals might occasionally move fast, usually this is as a result of one of “The 5 F’s” of Flight, Fight, Faint, etc of the “autonomous system” not the conscious system that can be thousands of times slower, or incapable of coming to a rational, reasoned response at all.

The apparant “logical choice” will be to build “synthetic autonomous systems” by use of AI…

For reasons that will not fit in a single post to a thread this AI-v-AI situation will not end well. In fact it is almost guaranteed to be a disaster much as ECM, ECCM, ECCCM and nearly all other “arms races” have amply demonstrated in the past.

Heck even the alleged fate of the Sabertooth Tiger demonstrates what an “evolutionary cul-de-sac” and many similar overly specific evolutionary behaviours will destroy you.

The worst enemy of survival and security is loss of generality to specificity.

AI will give lots of specificity with limited close in fuzzing, what it won’t as easily give is broad generality.

It’s Current AI of LLM and ML Systems Achilles heel…

Thus highly trained and skilled humans are still very much required to make the intuitive leaps that LLM Fuzzing just does not make.

Fedos • August 11, 2026 7:07 PM

Given that this story originates from someone with a financial interest in overselling the abilities of next word predictors, I’m taking this with a massive grain of salt.

Q • August 11, 2026 10:52 PM

A human user asks an AI to perform a task. The AI exploits a system vulnerability to carry out the request in an unethical or illegal manner that the user would not have authorized. Who is legally responsible? The foundation model? The open-weight user?

One can guarantee that the user will be held responsible. The big corps will make sure of that. They aren’t going to lose any time or money defending the little people.

So use these Awful Inference (aka AI) engines at one’s own risk. If it hacks into a system then the user will go to jail, not the big corps.

I’m sure the TOS will state that somewhere, buried in legalese in paragraph 13,678 section 49 part (f), or something.

Fresh flesh • August 12, 2026 12:02 AM

When I was young
I used to dream
And the wind blows
And the owl sings
And dogs are driven wild
And dogs break their chains
And run through the lands
A prey to madness
With wild eyes dying
With wild eyes burning
They raise their heads
They swell their cold necks
Like a cat that’s ripped its guts
Like a hungry child’s breath
Like a woman about to give birth
Like a young girl singing
At the stars in the north
At the stars in the south
At the stars in the west
At the stars in the east
At the moon
At the mountains
At the rocks
At the pain
At the thief
At the snakes

Reveal their black black backs
Fresh flesh
Glazed eyes stare
From long pale human faces
We cannot satisfy the hopes
We are now dead
We are all dead

yetAgain • August 12, 2026 12:17 AM

Yet again…
https://xkcd.com/416/

Winter • August 12, 2026 2:13 AM

If there is any vulnerability in anything, AIs are going to find and exploit them.

I think about this in a simplistic way:
That LLM has stored all stories written anywhere anytime in history. And all computer code and programs are nothing but stories.

The genius of LLMs is that the are able to store these stories as overlapping and branching story arcs. That way, there is a path from every story arc to almost every other story arc.

If you ask an LLM to write a story how to get into a gym class, it will draw on all the stories it has stored. It will always be able to generate a story about getting into a gym class.

If it doesn’t have your particular story it will generate one by combining parts of other stories. And another if this one doesn’t suit you, and another if you want, ad infinitum.

If you ask the stories to be written as computer and communication code and commands, it will do so. Programs and code are just another type of story.

In an agent, the LLM is inside a harness that will execute the story as code, and feedback the results to generate a new version, and repeat the cycle as often as you pay for.

Now we have a genetic algorithm:
Code -> Test -> Adapt -> go back

We know how well genetic algorithms work. Given enough iterations, they will find a solution if there is one.

It just seems like LLMs are rather efficient in generating plausible “stories” for genetic algorithms. Which is not surprising as they already store nearly every story and program ever written.

Lesson to learn:
If there has ever been written a story or program that tells how to solve your problem, or almost solve it, the LLM will know it and the harness, or agent, will find it.

Magnus • August 12, 2026 2:18 AM

“Our cyber defensive game has to be dramatically improved…very fast.”

Or, globally stomp on the AI companies. We (at least try our best) don’t let companies pollute the environment indiscriminately, kill whales, use slaves, or more-or-less lie in advertising.

And hound out of polite society anyone who uses AI. At the very least the individual concerned (if the story is true) should be kicked out of the gym. If not he/she should be shamed and ostracised, every other gym-goes leaving their sweat on equipment just before he uses it.

Clive Robinson • August 12, 2026 4:46 AM

@ Magnus,

With regards,

“Or, globally stomp on the AI companies.”

There is an apt saying about this,

You can not unring the bell, once it has been rung out.

The simple fact is once the weights for the DNN inside an LLM have been calculated by an ML System they are available to be,

1, Used in the DNN
2, Communicated from DNN to DNN
3, Stored in a new DNN.
4, Modified for use in a new type of DNN
5, Be updated with new data.

All without needing to start from raw input data.

In fact it is this that some in the US are claiming DeepSeek and other Asian / Chinese companies have done to make their DNN weight models (though reputable evidence is distinctly lacking).

Some of these models are claimed to run on “high end home systems” rather than large GPU arrays in data centers, so are in effect “portable”.

The other point is as @Winter has noted just above you,

blockquote>“If you ask the stories to be written as computer and communication code and commands, it will do so. Programs and code are just another type of story.”

<

blockquote>

The development of software which our Western World is now almost entirely reliant on is just a form of tool assisted “story telling” and always has been.

Mostly nobody really cares how mediocre the “software story is”, just that it is “Churned out quickly” and it certainly does not have to even function correctly…

Some have realised just how useful this makes LLM’s in Ralph Wiggum Loops running in Gas Town or other control frameworks, some of which like OpenClaw are themselves churned out by LMM.

‘https://devinterrupted.substack.com/p/inventing-the-ralph-wiggum-loop-creator

The fact is almost anyone who has the ability to break an idea down into parts can now use those as the descriptive building blocks for such an LLM system, thus create “software stories” that “suffice”.

The results may be an unholy mess but then people have claimed over the years that is,

“Microsoft management in action, producing junk code, by the truckload.”

And who is to argue with CVE scores.

As our host @Bruce has with others noted,

“The Genie is out of the bottle.”

And few want to put it back before they get their wishes…

So LLM usage inside armies of brain dead agents swarming at frenetic speed are here to stay. And even if made illegal here they will be used elsewhere with the resurgence of,

“The 1980’s and 1990’s ‘Make It So’ management style, spawned from Startrek Next Generation.”

Which appears to be back in “authoritarian command structures” of all types, even though it failed back as much as four decades ago.

But “each story told” brings with it “lessons to learn by” which means like it or not things will improve simply because “there will be less bad” at each iteration.

Consider that every human endeavor is

“A story in the making”

And you can see why some are asking the question,

“Even if we never get AGI, are LLM’s the tools of the new industrial revolution?”

It’s a question quite a few think the answer is “yes” to, and I can see why.

Others however think the same of Social Society, and well I ascribe to the view that the use of technology on society is generally not a good idea as history repeatedly shows. However it is starting to happen where authoritarian views and political mantras prevail, because of it’s “arms length” advantages.

Clive Robinson • August 12, 2026 5:33 AM

@ ALL,

I’ve mentioned the “Make if So” management style several times now and that it comes from Star Trek Next Generation that broadcast back in the 1980’s and 1990’s

And also that as a management style it was a failure.

I’m not going to go into why it was a failure, those who have experienced it in the past know why it was and why it’s generally a bad idea to revive it. Which unfortunately AI usage is doing currently at all levels. So graphic examples are abounding and will continue to do so untill people realise why it fails more than it works.

However people are acting like the Ralph Wiggum Loop in a Gas Town framework was something new and radical and thus special.

It’s not and I suspect few are prepared to admit that such things have been a staple of SiFi for quite some time.

In fact one good example of AI use for exactly this is from Star Trek Next Generation with Lt “broccoli” Barcklay going into the Holo-Suite to repair the Argus Array in the episode “The N-th Degree”.

If you watch it you will see the story from “April 1, 1991” covers everything we are currently doing with AI and some more that will no doubt happen fairly soon.

It will also give you a clue why “Make it so” fails in most cases. And where we are heading with such AI usage.

Hacketry • August 12, 2026 6:36 AM

My network faces an attack approx. every 2 seconds, 24/7.
I presume AI is behind the majority of those attacks.

AI Genie in the Wild

Comments

Leave a comment Cancel reply