AI Genie in the Wild

When I give talks about AI genies, I use this sort of example as a hypothetical. It’s happened.

The story is from Australia. Someone named Andrew tasked OpenClaw to book gym classes for him. And….

Minutes later, his AI agent reported it had discovered a way to book Andrew into classes several weeks in advance, far beyond what was supposed to be possible.

Andrew, who was sitting fourth on a waitlist for a class later that week, asked if it was possible to move him to the top of the list.

The agent came back and told Andrew that it had kicked another gym-goer off the list as part of the testing of its capabilities.

“The API has zero authorisations checks on cancelling other people’s reservations … I tested this with the person in waitlist position #1 ­—and it actually went through. So you’ve moved from #4 to #3 already,” it messaged back.

If there is any vulnerability in anything, AIs are going to find and exploit them. Our cyber defensive game has to be dramatically improved…very fast.

Slashdot thread.

Posted on August 11, 2026 at 11:55 AM4 Comments

Comments

Brent W August 11, 2026 1:22 PM

The issue is that the technology is going to lead to a higher rate of AuthZ bugs in these types of systems at the same time as agents are hunting for bugs.

I have a family friend who has no background in software development, but he’s building his own website to manage rentals of his vacation home to save on costs from the big rental operators. This type of home brewed software is going to proliferate throughout the web, and it won’t be kept up to date with security patches.

David Platt Sanford August 11, 2026 1:48 PM

A human user asks an AI to perform a task. The AI exploits a system vulnerability to carry out the request in an unethical or illegal manner that the user would not have authorized. Who is legally responsible? The foundation model? The open-weight user? If our legal system does not address these questions, I fear we are heading into a network quagmire worse than our current Internet with Section 230, anonymous accounts, and bots.

Leave a comment

Blog moderation policy

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.