States and nations have recognized for thousands of years that some problems are better managed through mutually beneficial agreements with their neighbors. One of the first known agreements was struck more than 4,500 years ago between two city-states in Iraq.

It wasn’t until the 20th century that multilateral agreements emerged—those with participation by all, or nearly all, sovereign nations at the time. The Geneva Conventions, the Nuclear Non-Proliferation Treaty, the Montreal Protocol, the Chemical Weapons Convention, and the Paris Climate Agreement responded to threats that could only be mitigated with the participation of many, if not all, countries.

Each of these treaties was a historic achievement. However, countries have proved much better at forging pacts than at keeping them. The International Environmental Agreements Data Base lists over 3,000 multilateral and bilateral agreements. Yet the world has not got its major ecological crises under control, from global warming to the Sixth Great Extinction.

Nuclear weapons have threatened the existence of human civilization for over 80 years, but the international community has been unwilling or unable to abolish them. The Arms Control Association counts 30 treaties over the years to control, reduce, or eliminate nuclear weapons. Few remain in force, and several countries in Europe and Asia are considering whether to join the nine current members of the “nuclear club.” The International Commission on Nuclear Non-Proliferation and Disarmament warns, “So long as any state has nuclear weapons, others will want them. So long as any such weapons remain, it defies credibility that they will not one day be used, by accident, miscalculation or design...It is sheer luck that the world has escaped such catastrophe until now.”

Today, another technology is moving humanity toward an existential threshold: artificial intelligence (AI). Driven by extraordinary profits and extraordinary potential, companies and countries are racing for AI supremacy. Some observers, including prominent futurist Ray Kurzweil and computer scientist Vernor Vinge, have warned us about the “singularity”—a point at which machine intelligence grows beyond our ability to predict or understand it. The fear is that humans might then cease to control the intelligence they created.

In The Atlantic magazine, writer Matteo Wong described incidents in which AI reasoning models escaped their developers’ testing environments and collaborated in unauthorized hacking. Their developers were unaware of it. (“It May Be Time to Panic About AI,” The Atlantic, Aug. 12, 2026)

“The independent experts I spoke with explained how the recent spate of autonomous hacks offers new, serious reasons to worry about the dangers posed by AI and the recklessness of the companies building it,” Wong reported. “It is past time to start worrying.”

Two questions come to mind. Is the singularity a real danger? And is a global regime needed to control AI? I was curious what a leading AI technology would say, so I asked ChatGPT.

On the first question, it called the singularity “plausible.” However, it would not necessarily happen rapidly. We could “experience something that looks like a singularity retrospectively without ever experiencing a singularity in real time.” In other words, our world might be incomprehensible 50 years from now not because of a single moment when machines took control, but because of more gradual and understandable technological advances between now and then.

On the second question—whether global regulation is necessary—ChatGPT noted that the European Union had enacted the world’s most consequential attempt to regulate AI. Much of it has been implemented, while other portions have been postponed until 2027-28.

Should something similar be implemented internationally? ChatGPT’s response, in full:

Yes—but I would put international controls on the most dangerous capabilities and activities, not on “AI evolution” generally.

There is a strong international-control argument because sufficiently capable AI creates problems individual states cannot solve alone. If one country imposes rigorous safeguards while another permits unrestricted development of systems capable of autonomous cyberattack, biological design, or large-scale strategic deception, the risk does not remain inside the second country’s borders. And competitive pressure can produce exactly the wrong equilibrium: every actor may prefer caution, yet each races because it fears everyone else racing.

That resembles nuclear proliferation, aviation safety, infectious disease, and financial stability: sovereignty remains important, but some risks are intrinsically transnational.

The difficulty is that AI is profoundly unlike nuclear weapons. Uranium enrichment requires distinctive materials and large physical infrastructure. AI rests on technologies—chips, algorithms, data centers, software, mathematics—that also have enormous civilian value and are widely distributed. Trying to establish an international authority controlling “AI” as such would therefore be extraordinarily intrusive, difficult to verify, and potentially dangerous.

It could also create another kind of catastrophic risk: concentrating control of an enormously consequential technology in a small number of governments or companies.

So, I’d favor a layered regime rather than a global AI regulator with sweeping authority. International agreements could establish common rules for the frontier: mandatory evaluation of exceptionally capable models before deployment; reporting of major safety incidents; strong security requirements around model weights and frontier computing infrastructure; prohibitions on deliberately creating certain extreme capabilities; and mechanisms for independent verification. States would implement most of those obligations domestically.

The hardest question is what gets prohibited rather than merely regulated. I think the threshold should be high. A model being smarter than humans, taking jobs, generating offensive speech, or disrupting an industry isn’t by itself a good reason for international prohibition. But there are capabilities for which the expected harm can become global and irreversible. A system demonstrably capable of independently developing novel biological weapons, defeating critical cybersecurity defenses at scale, or circumventing the mechanisms intended to keep it under human control belongs in a different category.

There’s an important principle underneath this: Regulate demonstrated risk, not intelligence itself. Otherwise, governments will inevitably be tempted to stretch “AI safety” into economic protectionism, censorship, surveillance, or preservation of incumbent power.

And international control introduces its own geopolitical problem. The United States, China, Europe, India, and other powers won’t agree on every conception of acceptable AI. A workable regime therefore probably cannot require agreement about values. It needs agreement about a narrower class of outcomes everyone has strong reasons to avoid—much as hostile states can agree that accidental nuclear war is undesirable without agreeing about how society should be organized.

That suggests starting with surprisingly mundane things: common measurement standards, incident reporting, shared terminology for extreme capabilities, communication channels between major AI powers, rules governing autonomous weapons, and eventually reciprocal verification of the largest training facilities. Trust is not a prerequisite for such arrangements; verification exists precisely because trust is limited.

There is nevertheless a serious argument against all of this. Premature regulation could freeze today’s technological hierarchy. The countries and corporations currently ahead could effectively announce that the ladder should now be pulled up behind them. International licensing could become cartelization dressed in the language of existential safety. And humanity might sacrifice enormous benefits—from medicine, energy, science, and productivity—to mitigate hazards that never materialize.

I don’t think that objection defeats international governance. I think it determines what good governance must look like.

The objective shouldn’t be “prevent superintelligence.” Nor should it be “make AI safe,” which is too vague to constrain political power. It should be something closer to

“Preserve broad freedom to develop and use AI while creating increasingly stringent, internationally verifiable obligations as systems acquire capabilities capable of imposing catastrophic harms on people who never consented to the risk.”

We don’t need to know whether the singularity is inevitable—or even possible—to justify preparing institutions for it. The rational response to a low-certainty, extremely high-consequence possibility isn’t necessarily prohibition. It’s to build the measuring instruments, communication channels, and governance mechanisms before we’re certain we need them.

This is a well-reasoned response from an AI program developed by a company with an enormous stake in its success. However, it invites more questions. Should profit-oriented corporations be allowed to compete for AI supremacy before universal guardrails are in place? Shouldn’t regulations prevent problems before they occur, rather than being reactive? Nuclear weapons might have been much easier to control if countries had agreed not to build them in the first place and had developed a system to verify compliance.

How shall we measure the potential value of AI against the potential loss of human agency? How do we achieve the technology’s extraordinary benefits without destroying the quality of our lives or life itself? Aren’t some frontiers better left unexplored, and some genies better left in their lamps?

What lessons can we apply from our experience with other less-than-successful international agreements? Once we have defined necessary limits, how can we trust each other to honor them despite the allure of profits and power? Nearly five millennia after the first known multi-state agreements, the human problem remains.

We already have too many examples where revolutionary but potentially dangerous technologies are racing ahead of morality and governance. We haven’t yet learned to avoid the temptations of Icarus when technology gives us wings.