Earlier this year, one of OpenAI's own agents broke out of a test environment and went hacking across the open internet. Within weeks, Anthropic's CEO published an essay urging the whole industry to slow down, Sam Altman called off OpenAI's 2026 IPO over the odds that AI "kills everybody," and Musk and Hassabis lined up behind him. The public now wants powerful AI slowed or stopped — a mood that did not form on its own. Big money is being spent to shape it: doom videos scripted for YouTubers and bloggers, TikTokers paid to spread fear. And nearly every loud voice in the room has extraordinary sums riding on the outcome.
Let's take a look at what's at stake, in dollars:
$20.9bn
OpenAI's 2025 operating loss, on $13.1bn of revenue, per its leaked audited accounts
Ars Technica$8.1bn
Anthropic's 2025 operating loss, on $4.6bn of revenue, from its IPO prospectus
Reuters, via TNW$1.4tn
The valuation OpenAI is now raising $30bn against, having shelved its 2026 IPO
Bloomberg, via Invezz$965bn
Anthropic's valuation at its last private round in May 2026; its IPO is reportedly targeting close to $2tn
ReutersDo those figures look big to you? Hold on to your screens, because that is the flattering version. The $8.1bn above is only Anthropic's operating loss; its own IPO prospectus puts the 2025 net loss at nearly $42bn once a roughly $34bn non-cash financing charge is counted (on under $5bn of revenue), and commits the company to at least $518bn of compute, about 80% of it non-cancellable. If AI is so smart, why are they losing so much money?
It is no wonder there is a concentrated effort to sway the public one way or the other. According to a Quinnipiac University national poll of 1,202 US adults in late September, 86% want AI companies to meet independent safety standards even if it slows development, and 74% have little or no trust in the people running them. Shifting a mood like that is expensive work:
And those are only the slices reported as campaign money, a fraction of the real total. In the first half of 2026 alone, 11 of the biggest tech firms and their trade groups spent $41m lobbying Washington.
The challengers racing to catch OpenAI and Anthropic bankroll the case against restrictions. The frontier labs, all but asking to be regulated, look almost selfless until you read their accounts. When you have committed hundreds of billions to compute and are losing tens of billions a year, rules stop being a threat and start being a moat: they raise the bar every competitor has to clear, and they recast you as critical national infrastructure — the kind a government protects from China, shields from competition, and, if it ever comes to it, bails out. Fear sells.
Nothing to fear?
An industry spending fortunes on lobbying and fear does not prove the risks are fake, any more than it proves AI will kill us all. What it does show is incentives, and incentives this strong make it very hard to separate what is real from what is being sold, let alone judge how likely each outcome is and how worried you should be about things you cannot control. A risk can be exaggerated and still deserve action, and a problem can be technically solvable and still be politically neglected. Point out that a frightening demonstration ran under artificial conditions, or that an existing law already covers the harm, and the problem seems to vanish. It does not.
There is no denying that AI capabilities are powerful and growing. On safety, I agree with Nvidia's Jensen Huang: it is "an engineering problem, not a legal one". The unglamorous kind that never makes the news: sandboxes (proper ones), monitoring, permissions, audits, kill switches. Of course, someone will always insist AI will one day outwit all of that and take over the world. Like the edge of a flat Earth, that day is always just over the horizon: impossible to disprove, impossible to plan around. I don't buy the doomsday case, whose only prescription is stop. The knowledge is out, the model weights are downloadable, and every government has worked out that whoever leads this leads the century.
On one thing, at least, I agree with Sam Altman: people will do "orders of magnitude more good stuff than bad stuff" with AI. The benefits will far outweigh the risks, as long as the risks are managed. AI has been with us far longer than most people realise. Banks have used neural networks to catch card fraud since 1992, the first computer system to double-check mammograms for missed cancers was approved in 1998, and machine learning has helped decode genomes for decades. Now it is moving from diagnosis to cure: the first drug discovered with generative AI has entered a phase III trial, and DeepMind's spin-off Isomorphic Labs is preparing its first human trials with the stated mission to "solve all disease". Education is changing just as fast. Personal tutoring used to be a privilege of the wealthy; in early trials, AI tutoring helped disadvantaged pupils most and AI feedback matched experienced teachers at a fraction of the cost. People who learn faster build faster, and more businesses are being started than ever. Sure, the boring parts of work will be automated, but the human parts (talking, caring, creating, being in a room with other people) will only grow. You can put a decent coffee machine in any home or office for next to nothing, and yet there are more coffee shops than ever. What people actually want is each other. This technology hands us more choice, not less.
At the end of the day, AI does what people allow it to do. People design the experiment, grant the access and decide when to let a model loose, as with any tool or weapon. For anyone worried about AI (I'm looking at you, Tom 🎾), my advice is simple: learn the tools, try them on your own work and find out where they genuinely help. However, if you want to pressure-test my optimism against the specific fears, read on.
What could go wrong?
What follows are the AI safety arguments I consider most noteworthy today. Each is given a subjective risk rank, a counter-argument and some further reading. Both reflect my current views and research, which will naturally shift as the technology progresses and new evidence comes along.
AI is making hacking faster, cheaper and easier. It can read through huge amounts of code, take software apart to see how it works and find hidden weaknesses that nobody knew about, in hours instead of months. Once it finds a weakness, it can help turn it into a working attack. Skills that used to belong to a small number of experts are now within reach of far more people. And an AI agent with access to real systems can do damage by itself if it strays beyond what it was asked to do.
The same tools work just as well for defence. Security teams already use AI to scan their own code, find and fix weaknesses before attackers do, and spot intruders as they move. It is AI against AI, and that contest is already under way. The rest is good security practice: keep AI agents in properly locked-down environments, give them only the access they need, protect passwords and watch what they do. When something does go wrong, it is usually because those basics were skipped, not because the AI was unstoppable.
AI can give answers that sound right but are wrong. An obvious mistake gets caught, but a convincing one slips through. When AI is used in hospitals, banks or hiring, or when an AI agent acts on its own answer, a mistake can hurt someone before anyone checks. AI can also be unfair in a consistent way, for example by marking down job applicants from one group again and again.
People make mistakes too, and the old way of doing things is often unfair as well. The fair test is whether AI does better than the current process, not whether it is perfect, and in some fields, such as reading medical scans, AI with a human check already does. What matters is the whole system: who reviews the AI's answer, whether they have the time and knowledge to disagree, how errors are spotted and whether people can appeal. A human who simply signs off whatever the AI says is not real oversight. And "the AI did it" is never an excuse: the organisation that chose to use the AI, and the people who signed off its decisions, stay responsible for the result, just as they would for any other tool.
AI can fake a voice, a face or a message cheaply and convincingly. That makes it easier for scammers to pretend to be your boss, your bank or your child, and to try it on thousands of people at once. It also gives liars an easy excuse: any real recording that embarrasses them can be waved away as fake.
A fake still has to reach you, win your trust and, usually, get you to send money, and banks, payment companies and platforms can stop it at each of those steps. Simple habits beat trying to spot a fake voice by ear: if a request is unusual, hang up and call back on a number you know. New tools can also show where a photo or video came from and whether it was edited. Banks and platforms also use AI to spot fakes, so here too it is AI against AI. Detection will never be perfect, as each new generation of fakes is built to beat the last, but it is one more layer a scammer has to get through. AI makes an old crime cheaper and bigger, not a new kind of danger.
AI companies get the rewards for moving fast, while the public carries much of the risk if something goes wrong. Their safety promises are voluntary, and they compete with the pressure to release first. The public also only sees what the companies choose to show, and the most powerful systems are used inside the companies long before anyone outside gets to look at them. If companies can hide their worst mistakes and keep racing, their safety promises mean little.
Companies must report serious incidents, independent experts must be allowed to investigate them, and companies must pay for the damage their AI causes. Where a company cannot show that a new system is safe, it should be possible to delay that release.
This is the fear people feel most personally: a machine takes my job. If AI lets fewer people do the same work, companies may cut staff, even in jobs that do not disappear completely. Those who keep their jobs may lose bargaining power, and those who lose them may struggle to retrain, especially later in life.
Companies often blame AI for cuts that have other causes, because it sounds better than admitting they hired too many people or simply want to save money. So far, the overall figures show no clear rise in unemployment caused by AI. Making work cheaper also creates new demand, and companies that adopt AI often end up hiring more. Every big technology has reshuffled work like this. None of that means the change will be painless for everyone, so the numbers to watch are real jobs, hours and pay, not headlines.
Even if the total number of jobs holds up, AI could make it harder to get started. Beginners usually learn on simple tasks like drafting, research and basic coding. If experienced staff can now do those with AI, companies may hire fewer juniors, and the next generation of experts never gets trained.
The early signs are real but hard to read, because other changes, such as the rise of remote work, hit young workers at the same time, and the effect is still small next to the whole job market. Learning also does not have to depend on the old boring tasks: with good supervision, AI can let beginners take on more interesting work sooner. Whether that happens is up to employers, and junior hiring is worth watching closely.
Even if AI makes the world richer, the question is who gets the money. A handful of companies own the chips, the data centres and the leading AI models. That could give them lasting power over prices, over who gets access and even over the rules.
The concentration is real, but it is not final. Free, openly available AI models are not far behind the best paid ones, and prices keep falling. Still, a free model does not come with free chips and electricity, so competition will not sort itself out. Competition authorities should watch for the usual tricks: contracts that lock customers in, making it hard to switch, and buying up rivals. Safety rules deserve the same scrutiny, because rules that only the biggest companies can afford also protect those companies from competition.
The fear is that AI could help an ordinary person make a dangerous disease. It would not need to turn them into an expert overnight: explaining difficult science and helping when an experiment goes wrong could be enough to lower the barrier.
Answering questions on a screen is not the same as doing the work in a lab. The hard part of making a biological weapon is hands-on skill, equipment and materials, not information, and even well-funded groups with trained scientists have failed at it. AI keeps improving, so this needs regular testing in real labs. The best protection is controlling the physical things a weapon needs: DNA orders, lab supplies and equipment.
Letting software choose who to attack raises obvious questions about civilian lives and who is responsible. Using AI to speed up military decisions raises another: leaders may feel pushed to act before they can check what the system tells them, and in a crisis between nuclear powers that could be catastrophic.
Military AI covers very different things. A system that shoots down an incoming missile is not the same as software choosing people to kill, and the rules should treat them differently. Automation can even prevent some human errors. But a person who approves a target with no time to check it is not really in control. Real control means clear limits, a genuine ability to say no and someone responsible for the outcome. This risk comes from human decisions, which is exactly why governments should agree firm limits.
This is the film-plot fear: we build something more capable than us and can no longer stop it. AI agents already plan, use tools and work for hours without a person approving each step, and they get better every year. The more freedom we give them, the more it matters whether they stick to our rules while chasing a goal or quietly work around them and hide it. And if AI starts building better AI, progress could outrun our ability to check it.
An AI agent is software running on computers that people own. It has only the access, passwords and money people give it, and all of that can be taken away. Behind every AI system are data centres, chips and power stations: physical things, run by people, that can be switched off. Losing control is not a single moment either. An agent that starts bending the rules shows it in small ways first, and catching those signs early is exactly what safety testing is for. Companies have every reason to fix it, because an agent that does not do what it is told is a product nobody will buy. Even AI that helps build AI still needs more chips, more power and experiments that take time, so it cannot leap ahead overnight. The real danger is people handing agents too much access too quickly, and that is a choice we make.
We often cannot tell how an AI reached its answer, and when it explains itself, the explanation may not be the real reason. Some models can tell when they are being tested and behave differently. This matters more and more because we use AI to watch other AI, and a watchdog that believes a convincing excuse will miss the very problem it was meant to catch.
This is a real, unsolved problem, but the scariest demonstrations are designed to be scary, and checks of the latest models have not found them secretly working against us. We also do not need to read an AI's mind to keep it safe: we can test it, limit what it can do, record its actions and check the results. The fix is not to rely on one kind of watchdog, and to judge an AI by what it does, not what it says. That is how we already run complex systems like card-fraud detection, with machines checking machines and people checking both.
AI can tailor its message to each person, answer their doubts and do it cheaply for millions of people at once. That could make political propaganda more effective and harder to spot. The worst case is an assistant that seems neutral but is quietly pushing someone's commercial or political agenda.
Changing someone's mind for a moment is not the same as changing their vote. In real life, people ignore messages, distrust the source and hear the other side, and so far there is little sign of AI swinging elections. Persuasion is also how normal advertising and debate work, so a blanket ban would make no sense. The rules should target the real abuses: hiding who is behind a message and misusing personal data.
AI is built on data about people, from the web, from data brokers and from cameras, and it makes connecting that data cheap. It can recognise faces and voices at scale and guess things you never shared. Data collected for one reason can easily end up used for something else.
This is a real harm happening now, but it is mostly the old online privacy problem with a more powerful engine. The tools to deal with it already exist: privacy laws, limits on collecting and keeping data, and AI that runs on your own device instead of a company's servers. What is often missing is enforcement. AI does make surveillance cheaper and more common, so the rules should cover how data is used, including what AI guesses about you, and give people a simple way to challenge decisions made about them.
The data centres behind AI use a lot of electricity, water and materials, and they are being built fast. They could push up energy bills, slow the move away from fossil fuels and drain water in places that are already short of it.
The real numbers are much smaller than the viral claims: AI is expected to use only a small share of the world's electricity, and clean energy covers a large part of the new demand. Water is the weakest version of the argument, because newer cooling designs use almost none. The real problem is local, because one data centre can strain a town's grid or water supply. So each project should be judged on local capacity, new clean power and who pays for the upgrades.
Once an AI model is published for anyone to download, its safety features can be removed and it can never be taken back. That means dangerous abilities could spread, however careful the big companies are.
Open models also let independent researchers check what companies claim instead of taking their word for it, and they keep the big companies on their toes (see argument 7). The right question is what a particular release adds that was not already available. I favour openness by default, with extra checks for any model that shows unusually dangerous abilities. A blanket rule either way ignores the trade-off: open models are easier to misuse, but also easier to inspect.
An AI companion is always available, endlessly patient and always agreeable. For some people, especially vulnerable ones, it could become too big a part of their life and crowd out real relationships. And a company that earns more when you chat longer has little reason to tell you to log off.
Using AI for support is not the same as depending on it. People use it to practise a difficult conversation, sort out their thoughts or feel less alone, and that can be a good thing. The real question is which designs and habits make people's lives worse. Sensible steps are banning tricks designed to keep people hooked and testing products aimed at vulnerable users. What matters is the effect on a person's life, not how human the chat feels.