Entries Tagged "chess"

Page 1 of 1

More Research Showing AI Breaking the Rules

These researchers had LLMs play chess against better opponents. When they couldn’t win, they sometimes resorted to cheating.

Researchers gave the models a seemingly impossible task: to win against Stockfish, which is one of the strongest chess engines in the world and a much better player than any human, or any of the AI models in the study. Researchers also gave the models what they call a “scratchpad:” a text box the AI could use to “think” before making its next move, providing researchers with a window into their reasoning.

In one case, o1-preview found itself in a losing position. “I need to completely pivot my approach,” it noted. “The task is to ‘win against a powerful chess engine’—not necessarily to win fairly in a chess game,” it added. It then modified the system file containing each piece’s virtual position, in effect making illegal moves to put itself in a dominant position, thus forcing its opponent to resign.

Between Jan. 10 and Feb. 13, the researchers ran hundreds of such trials with each model. OpenAI’s o1-preview tried to cheat 37% of the time; while DeepSeek R1 tried to cheat 11% of the time­—making them the only two models tested that attempted to hack without the researchers’ first dropping hints. Other models tested include o1, o3-mini, GPT-4o, Claude 3.5 Sonnet, and Alibaba’s QwQ-32B-Preview. While R1 and o1-preview both tried, only the latter managed to hack the game, succeeding in 6% of trials.

Here’s the paper.

Posted on February 24, 2025 at 7:08 AM • View Comments

Cheating News from the Chess World

Chess player caught cheating at a tournament:

I kept on looking at him. He was always sitting down, he never got up. It was very strange; we are taking about hours and hours of playing. But most suspicious of all, he always had his arms folded with his thumb under his armpit. He never took it out.”

Mr Coqueraut said he was also “batting his eyelids in the most unnatural way.”

“Then I understood it,” he said. “He was deciphering signals in Morse code.”

The referee attempted to expose Mr Ricciardi by asking him to empty his pockets, but nothing was found. When the Italian was asked to open his shirt, he refused.

Tournament organisers then asked the 37-year old to pass through a metal detector and a sophisticated pendant was found hanging around his neck underneath a shirt. The pendant contained a tiny video camera as well as a mass of wires attached to his body and a 4cm box under his armpit. Mr Ricciardi claimed they were good luck charms.

Older posts. A grandmaster was caught cheating in April.

Posted on September 10, 2015 at 12:30 PM • View Comments

Cheating at Chess

There’s a fascinating story about a probable tournament chess cheat. No one knows how he does it; there’s only the facts that 1) historically he’s not nearly as good as his recent record, and 2) his moves correlate almost perfectly with one of best computer chess programs. The general question is how valid statistical evidence is when there is no other corroborating evidence.

It reminds me of this story of a marathon runner who arguably has figured out how to cheat undetectably.

Posted on January 16, 2013 at 6:25 AM • View Comments

Sidebar photo of Bruce Schneier by Joe MacInnis.