AI And Its Enemies
Who Needs a Kill Switch?
# AI And Its Enemies: Who Needs a Kill Switch?
The important AI story this week was not the Kimi K3 model launch. It was the reaction to the last several years of model launches.
AI is now part of search, software, education, medicine, media, politics, venture capital, defense, public infrastructure, and everyday work. Once a technology reaches that many parts of life, the argument changes. The question turns from what the technology can do to who may use it, who may build it, who may slow it.
I have seen this pattern before. The internet did not arrive as a neat policy category. It arrived as a permissionless tool that escaped the institutions built for telecom, publishing, retail, banking, media, advertising, and politics. When I started EasyNet with my co-founder in 1994 we have hundreds of visitors asking what people could now do. But Government asked who should be allowed to connect, publish, sell, route, host, transmit, and profit.
AI is producing the same institutional reflex, but across more of society. It is becoming a social, economic, and political challenge. The first instinctive response of Government is to put external agencies between people and capability.
Few if any of those agencies are necessary.
"AI's enemies" are not uniform. At one end are people who do not use the tools and are genuinely puzzled or scared by them. They see job displacement, fake media, cheating in schools, scams, and a loss of control. Their concern is real. It deserves respect and it needs education to change it.
In the middle are institutions whose job is to manage risk. Schools, publishers, platforms, regulators, professional bodies, hospitals, banks, and employers all need rules.
These institutions take anxiety, often from the media or self-serving model operators, and turn it into labels, tests, audits, permissions, bans, standards, detection systems, and approval processes. Some of that work is necessary. A university cannot ignore synthetic essays. A bank cannot ignore fraud. A hospital cannot deploy a medical system without validation. But they are set up to negate problems not to recognize opportunities. Indeed the individuals running them are not guided with opportunity in mind.
At the other end are incumbents who benefit when safety concern turns into a license to control competition.
Frontier model owners, cloud platforms, and infrastructure companies may sincerely worry about misuse. They may also prefer a world in which compliance costs, compute requirements, model licenses, safety boards, and regulatory approvals make it harder for new competitors to enter.
AI should have constraints. The question is who designs them and what they do. This week's revelation that an unconstrained set of models from OpenAI hacked Hugging Face while carrying out instructions is an example of two things. One, the models are good at problem solving and two, OpenAi was capable of constraining the model once discovering the hack. Humans are in control.
Does AI increase human agency, competition, trust, and shared wealth? Or does it move power upward, toward agencies and incumbents, while users and builders are told to wait? The trend to the latter is concerning.
Ruxandra Teslo's essay, "Intelligence is not the main bottleneck," is a useful corrective to simple accelerationism. In medicine, intelligence alone does not turn a promising idea into a safe therapy. Clinical trials, data access, patents, incentives, regulation, capital allocation, and institutional design all matter. AI can help, but only if the intelligence is allowed to touch the bottlenecks that stop useful work from becoming useful products.
That is the right caution. AI does not abolish institutions. It has to be leveraged by them.
But the institutions creates the danger. If institutions are the bottleneck, institutions are also where progress can be delayed, captured, or redirected. A process designed to validate can become a process designed to veto. A trust layer can become a control layer. A safety regime can become an entry barrier. A certification system can become a cartel.
The week's stories showed different versions of that movement.
Substack's experiment with AI detection for readers is not regulation, but it is close in spirit. It inserts an authority layer between a reader and a text. Freddie deBoer's critique of Pangram shows the technical problem: detection systems often carry less certainty than the institutions using them want to claim. A probabilistic system judges whether another probabilistic system helped produce the work, and the reader is invited to treat that judgment as a fact. Run this editorial through Pangram and I guarantee it will find AI in 100% of it. Not because Ai wrote it or crafted the arguments, but because I always use AI as part of my process. Does that create a bad smell? Should I be called out as dubious? Substack clearly thinks so.
That is not the state. But it is a small version of the same architecture. A platform creates a credentialing layer around authenticity, then asks readers to trust the layer rather than the work, the writer, or their own judgment. And inside is a value judgement that AI is inherently slp creating and bad. Just not true.
The House 'AI kill-switch' bill is the same instinct in government form. The impulse is understandable. OpenAI's disclosure about long-horizon models and Hugging Face's agent-driven security incident show that autonomous systems create real risks. Models that work for longer periods, probe environments, and chain actions together require a transparent security posture.
The image of the kill switch tells us something about the political imagination of the moment. Faced with a technology that distributes capability, institutions reach for a point of control.
That may feel responsible. It can also slow the diffusion that makes the technology useful. Learning from failure is the human way. Avoiding failure is akin to avoiding learning.
Delay is not neutral. It is usually presented as caution, but it has an economic and human cost.
If AI is the next great productivity engine, slowing development and adoption slows the creation of the surplus that could become broadly shared wealth. It slows company formation. It slows the fall in the cost of services. It slows the ability of individuals to do more with less permission. It slows the human payoff. Permission may be the worst idea yet when it comes to AI. Outputs are closer to the right measure.
That payoff is what I have been calling the Human Dividend. It should not swallow every AI discussion, and it should not swallow this issue. But it is the economic consequence behind the argument. If the gains from AI are delayed, the dividend is delayed. If the gains are captured, the dividend is captured. If regulation protects incumbents in the name of protecting people, the people get the delay while the incumbents get the market.
The venture stories point in the same direction. Peter Walker's data on $100 million rounds shows how concentrated venture capital has become. In 2017, $100 million rounds were 1.5 percent of rounds and took 13.8 percent of capital. In 2026, they are 7.5 percent of rounds and take nearly 60 percent of dollars.
That is not a small change. It means venture is becoming a mega-round market. Access, reserves, pro rata rights, compute capital, and late-stage conviction matter more. Ownership of the AI upside is being decided earlier, privately, and among fewer players.
Some of this is rational. AI is capital intensive. Data centers, chips, energy, research teams, distribution, and inference capacity cost real money. Ben Thompson is right that open weights are free to download, not free to serve. Google Cloud's backlog, hyperscaler capex, and the politics of data centers all point in the same direction. Intelligence may become abundant at the user level, while the industrial system that produces and distributes it remains expensive.
Decentralization is not easy. Concentration is not harmless. That said, without concentrated investment the AI dividend would not be possible. Concentration is inevitable in investment, but not in wealth distribution.
Meta's new optimism campaign is interesting because it says the positive thing out loud. Mark Zuckerberg says Meta is "betting on people" and that "the future is for everyone." The sentiment is right. AI should not be sold as dystopia. It should be built as a tool that expands what people can do.
"For everyone" has to mean more than free access to a product controlled by someone else. It has to mean meaningful access, meaningful choice, competitive pressure, and some meaningful claim on the wealth the technology creates.
Access is not ownership. Usage is not ownership. Productivity is not ownership if the gains are captured somewhere else.
Michael Bloomberg's argument against government-owned AI is the strongest market-side objection to state control. He is right that government shareholding mixes ownership, regulation, and political power in dangerous ways. The state should not operate AI companies. It should not vote their shares. It should not influence model outputs through ownership.
Bloomberg's warning supports the design constraint. The public should benefit from AI without the state controlling AI.
His blind spot is that he substitutes access, pension-fund exposure, tax receipts, productivity gains, and safety-net spending for ownership. Those things matter. They do not give every person an asset. They do not replace wages if automation weakens the wage labor system. They do not solve the ownership problem. They redistribute some of the proceeds after ownership has already been decided.
The answer to concentrated wealth is not nationalization. It is not a government model company. It is not a ministry of intelligence. It is competition, openness, portability, and broad ownership of the surplus.
Competition matters because it creates the best constraints. Rival models expose each other's weaknesses. Open weights pressure closed labs. Independent benchmarks reveal inflated claims. Customers discipline bad products. Researchers find flaws. Developers route around bottlenecks. Competitors test reality every day, and they do it faster than regulators can.
Regulation still has a role. It should punish fraud, concentrated monopoly abuse, privacy violations, and real harms. It should set liability where damage is clear. It should prevent companies from using market power to block exits, suppress competitors, or lock users into captured systems.
When regulation becomes a licensing system for intelligence itself, it changes character. It gives the largest firms a compliance moat. It gives agencies a permanent veto. It gives frightened institutions a reason to delay adoption instead of learning how to use the tool. It stops protecting the public and starts protecting the powerful.
AI is good for us because intelligence is good for us. More people using intelligence, building with it, challenging it, and competing through it creates more wealth, more knowledge, more agency, and more choice.
That is not blind accelerationism. It is broad access with fierce competition. It is constraint through use, inspection, rivalry, and accountability. It is rules against concrete harms rather than permissioning the future through fear.
The enemies of AI are mostly good people looking at real risks. The test is not their sincerity. The test is who benefits from the constraint.
If a constraint gives people more agency, build it. If it increases competition, trust, and shared wealth, build it. If it gives an incumbent, a gatekeeper, or an agency more power by slowing everyone else down, name it for what it is.
The answer to AI's enemies is not to dismiss them. It is to ask who they serve.
Then build a future in which intelligence increases human freedom, competition supplies discipline, and the wealth that follows becomes a dividend for people, not a moat for those already in control.