Prospects for an AI Treaty

During the U.S.-China meetings last week, one of the key topics was AI, and a "Dialogue" was established as part of this. But many people want broader U.S.-China cooperation on AI, with a few calling for an AI treaty of some sort. What are the prospects for such a treaty? For the reasons set out below, while I think a treaty would be useful, I am skeptical that we will see one in the next couple years.

Domestic regulation should come first

For governments to be in a position to negotiate an AI treaty, domestic policy and regulation needs to be sorted out first, and I don't think we are close to that at the moment in the U.S.

As most of this blog's readers probably know by now, I worry about regulation being ineffective or counterproductive, although my views vary depending on the specific issue. With social media, for example, I am a skeptic of regulation. Many of the proposed regulations I've come across seem either pointless or harmful, and the problems can be more effectively addressed through private sector development of alternative social media apps/websites. Regulation isn't going to make the existing walled garden platforms much better, and in my view the best approach is for everyone to shift away from those to the emerging alternatives.

On AI, though, I think extensive regulation is needed, as there are a wide range of potential harms from AI products. As much fun as it can be to assign a somewhat arbitrary % figure to the chances of humanity being destroyed by AI, I'm less worried about that than I am about more basic things such as hacking or the psychological impact of AI chatbots. There are also many uncertain legal issues related to AI that would be nice to get some clarity on, and regulation could provide that.

Unfortunately, it seems to me that we are unlikely to get significant federal regulation on these issues while Trump is in office. My sense from the things Trump has said is that he does not want regulation in this area, and from what I can tell most people in the Trump administration seem to agree, or at least will go along with his view. As a result, any actions they take are likely to remain limited in scope and non-transparent, which is where things stand now. I doubt they can be swayed on this, and barring some major AI-related catastrophe that forces a response, I don't see anything happening here. Even the biggest blue wave I can imagine in the midterm elections won't give the Democrats a veto-proof majority in Congress, and thus I will be surprised if there is any federal AI regulation other than ad hoc White House guidance for the foreseeable future.

In the meantime, people can and should prepare for future legislative and regulatory action. Members of Congress can discuss the issues and make proposals, and various states will do some regulating. In this way, the next couple years can be used to map out what kind of regulation would be useful, so as to have something ready when the political situation allows it to be enacted at the federal level.

AI treaty talk

In the midst of all this non-action on federal regulation, and with President Trump meeting with President Xi last week, some people were calling for the negotiation of an AI treaty. Here are a few examples:

• Senator Bernie Sanders said: "With the future of humanity at stake, Trump must negotiate a comprehensive treaty with President Xi of China to establish a pause on advanced AI development and a ban on super intelligence."
• Dario Amodei of Anthropic said: "we should also aim for a worldwide pacing of the frontier ... Global pacing will require cooperation with China, the autocratic country with by far the most advanced AI capabilities."
• A group of Democratic Senators wrote a letter to Trump calling on him "to negotiate a formal agreement between the U.S. and China on guardrails and global standards for the development, testing, and use of frontier Artificial Intelligence (AI) models with Chinese President Xi Jinping."

During the U.S.-China talks, AI was, in fact, one of the issues on the agenda. Not surprisingly, though, very little was achieved. A White House fact sheet – which uses Trump's term for AI – says:

The two countries established the U.S.-China Super Intelligence (SI) Dialogue to exchange views on risks and benefits related to SI. The next exchange will occur by November 2026. The United States and China also agreed to establish a bilateral communication channel for SI incidents.

My sense is that a U.S.-China Dialogue, along with a channel for notifying AI-related incidents, will have a very limited impact on the regulatory issues here, and I am skeptical that any of this will lead to a significantly broader AI agreement in the near future.

This is unfortunate, because an AI treaty would be a good idea. Every government will be dealing with various problems related to AI, and coordinating these efforts would be useful.

But in the absence of a robust federal regulatory framework in the U.S., I can't see progress on an AI treaty happening anytime soon. In a situation where there is limited federal regulation, and the Trump administration doesn't want to pursue anything other than its own limited oversight of these issues, I don't know what a treaty would say other than "governments can do pretty much whatever they want." That's not particularly helpful when governments are trying to figure out what needs to be done and how to do it in a cooperative way. As far as I can tell, then, while an AI treaty with the U.S. and China as the key actors could be useful, it's going to have to wait for a new administration.

To be clear, even with a federal regulatory framework in place, such a treaty will be a challenge to negotiate, as the specific concerns of China and the U.S. – and of other countries as well – vary quite a bit. Nonetheless, once the U.S. has worked out its own views on AI regulation in a post-Trump world, I think it's worth sitting down with other governments and trying to reach common ground on international rules that could help guide everyone's domestic regulation.