Essays in the Category "AI and Large Language Models"
Page 1 of 11
AIs as Modern Genies
With AI we have built genies. Old stories told us how this goes.
In April, an artificial intelligence (AI) agent conducting a routine task at a company hit a snag, tried to solve it, and soon ended up deleting the company’s database along with all of its backups. In July, OpenAI asked an unreleased AI model to attempt a hacking test. Instead of staying in the isolated box the developers had put it in, the model hacked onto the open internet and into another company to steal the answers. And as reported in August, an AI agent booked someone into a full gym class by figuring out how to cancel other people’s reservations. In all three cases, the AI completed the task it was given—but in ways that ran counter to its controllers’ intentions…
No, AI Doesn’t Mean the End of Mathematics—at Least Not Yet
Mathematicians are raising concerns that the technology could kill their profession. But they still have abilities AI doesn’t
Earlier this month, about 40 top mathematicians gathered at OpenAI’s offices to discuss the future of their profession. The meeting was off-the-record, but if recent articles by mathematicians are any guide, it was mostly pretty glum. People fear for their jobs, their careers and the work they love.
We think the contrary view is more likely, at least in the short-term. AI models are nowhere near as capable as experienced academic mathematicians.
This isn’t to say that AIs aren’t producing stunning mathematical results at the level of PhD researchers. In mid-May, OpenAI …
If the Markets Reject OpenAI and Anthropic, the US Should Nationalize Them
From space to telecommunications, the US has a long history of fostering technology for the public good. These AI models could be aligned to democratic values, not corporate profits
OpenAI, and then Anthropic, were each formed by AI developers who feared unrestrained corporate AI development—specifically, that companies like Google and Meta would steer the technology towards deleterious, maybe even catastrophically unsafe, outcomes for society. Their founders proclaimed that their new labs, uniquely, could be trusted to develop the technology in humanity’s best interest. But each, in turn, were themselves co-opted by the same market incentives, themselves becoming corporate behemoths zealously guarding future investor value rather than the public interest…
Rewiring Democracy: Citizen Science Leads to Civic AI in Scotland
This is the fourth in a multi-part series by Sanders and Schneier going into depth on real-world examples of democratic technologies from their book, Rewiring Democracy: How AI Will Transform Our Politics, Government, and Citizenship. Their first piece was about the Japanese digital democracy party "Team Mirai," their second was about the Swiss Public AI model "Apertus," and the third was about the civic technologists of Open Knowledge Brazil.
A generation ago, only a select few—a scientific elite—got to participate directly in the discovery of new astrophysical phenomena like exotic galaxy types or new worlds around other stars using the latest data from space telescopes or powerful observatories. This work was rarefied because the mechanics of astronomical observation were complex, the ability (and willingness) of scientists to distribute that data was low, and the capability of non-experts to understand and provide meaningful input to the science was thought to be very limited…
Separating AI’s Technological Problems from Its Capitalism Problems
AI represents the first time we humans can do cognitive work outside of our bodies at scale. The only comparable moment is the early years of the industrial revolution, when new technologies like the steam engine provided a quantum leap in our ability to do mechanical work outside of our bodies at scale. If AI’s cognitive capabilities become integrated into our lives, businesses, and governments—a process that will take years if not decades—society will be as unrecognizable as the modern world would be to a preindustrial farmer. And yet, Americans—by a wide margin—…
The OpenAI Hack Shows the Genie Is Out of the Bottle
Attempts at control are futile. Policy should now turn to defense.
Earlier this month, two of OpenAI’s models broke out of their containment sandbox and attacked another AI company. The story is kind of wild. OpenAI was running security tests on two of its models: GPT-5.6 Sol and an unreleased model that is almost certainly GPT-6. In particular, it was running the ExploitGym benchmark, which measures how good a model is at turning security vulnerabilities into working exploits: basically, offensive cyberattacks.
Since these were internal tests, OpenAI locked those models in a secure sandbox that denied them access to the internet. But it was running the models without any safety filters that would prevent them from offensive cyber-actions. That meant that there was nothing to prevent the models from trying to …
How Do We Prevent AI Agents from Going Rogue? It Starts with a New Kind of Measurement
Like genies of folklore, AI agents take their instructions literally – to potentially disastrous effect. We must track their ability to do what we actually mean
In July, Hugging Face, a company that hosts much of the world’s AI software and open-source AI models, was hacked. A malicious dataset had been used to run code on one of its servers. Whoever was behind it captured internal security credentials and moved through systems over a weekend, running thousands of actions from a swarm of temporary server environments. It looked like the work of a sophisticated criminal group.
It was not. It was one of OpenAI’s new, still unreleased GPT models.
Their science experiment had escaped the lab. OpenAI was running the unreleased AI model through a benchmark that tests how well AI can successfully hack systems. To push the limits and evaluate the AI’s true capability, the company switched off the safety filters that normally stop it from doing this kind of hacking. Aware that this could go wrong, they confined the AI to an isolated environment and denied it access to the internet…
Should You Use AI for a Task? Here’s a Simple Way to Decide
Sometimes, what matters isn’t your output but what you put into the process. Think of it like work vs. the gym
I teach public policy at the Harvard Kennedy School and the Munk School at the University of Toronto. And it will come as no surprise to you that my students regularly use AI to complete their writing assignments. Doing so is a waste of their tuition money. But if their entire career is going to include AI writing assistants, why shouldn’t they embrace their future?
The best way I’ve found to explain the dilemma comes from the AI researcher Daniel Meissler: it’s the difference between work and the gym.
At work, if your job is to move a bunch of heavy things from one side of the room to another, you should use whatever assistive tech you have on hand: a wagon, a forklift … even an AI-powered robot. But at the gym, it makes no sense for that robot to lift weights for you. The point of weightlifting isn’t to move heavy things across the room; it’s to actually lift those heavy things…
Why AI Needs a “Genie Coefficient”
Proposing a new metric for whether AI does what you actually want
Major benchmarks measure what AI can do. None measure whether it does what you mean: the distance between what you ask an AI to do and the unspoken assumptions about how you want the AI to do it. We propose a new metric: the Genie coefficient.
There’s often a gap between one person’s request and another’s understanding. Most of the time, we bridge it using general knowledge. For example, if you ask a friend to get you coffee, they’ll pour a cup from the pot or buy one from a coffee shop. They won’t bring you a bag of raw beans or snatch a cup from a stranger and hand it to you. You never specified any of this. You never had to…
The Fight Against AI Data Centers Is Important—but It’s Just a Starting Point
AI companies want to capture the value created by entire industries. That concentration of wealth and power is society’s greatest risk
Opposition to AI data centers has emerged as a primary theme in US politics, one that—surprisingly—doesn’t fall along party lines. We applaud people coming together for constructive debate on any issue, and agree that communities need to evaluate whether any economic benefits these data centers bring is worth their costs. Still, we worry that a focus on data centers obscures the larger impacts of AI on people’s lives: the concentration of power of AI companies, and their widespread political and financial influence.
Local data center opposition is grounded in legitimate concerns about misallocation of land resources when housing is at a premium, …
Sidebar photo of Bruce Schneier by Joe MacInnis.