Leaps of Faith Become Harder over Time
... whether you're an iguana, or the USA trying to get China or France to cooperate on AI.
In any field, cooperation is hard. And if the stakes rise over time, cooperation becomes harder over time. The field of AI is no exception.
To reduce extinction risk from powerful AI, countries around the world need to cooperate and opt out of the race of building ever stronger AI.
Inconveniently, being ahead in AI is likely massively beneficial for individual countries in the short-term. It could strengthen a country’s geopolitical power position, its economic growth trajectory, put its defence capabilities ahead of other countries, and give its intelligence community an edge. However, having conditions that incentivise countries to be ahead of the pack as far as possible in AI is bad for everyone in the long term, because it increases extinction risk from unaligned, out-of-control AI by making actors prioritise progress over safety. But because of the short-term benefits of staying ahead in the AI innovation race, we need strong international cooperation which reassures each country that others won’t forge ahead unilaterally.
So how do you get competing actors to cooperate in a low-trust environment? One way of going from competition to cooperation requires one side to start a trust-building process by demonstrating trustworthiness through a costly signal. Essentially, one country needs to take a benevolent leap of faith and step back from competing, through which they open themselves up to being exploited or left in the dust by its competitors.
In AI this costly signal, this leap of faith, would mean that one country unilaterally opts out of the race towards building more powerful AI by verifiably hampering its own ability to compete. Such a leap of faith could look like banning AI training runs over a certain size, or restricting the size of compute clusters in the country. Of course these signals are especially powerful if taken by a major AI power, like the US.
The snag is that in AI, taking a leap of faith becomes harder over time. The longer countries wait and pursue their own national interests, the more difficult and risky it will become for any side to take a leap of faith to boost international cooperation. As AI systems improve, not only do the collective stakes rise (it becomes more likely that an unaligned AI is released and causes the extinction of humanity). The individual stakes for countries and AI companies rise as well (the further AI progresses, the more they are threatened by every advancement in AI technology that their rivals gain). Therefore, as time progresses and AI systems get more powerful, taking a leap of faith becomes ever more risky and more costly:
Because AI technological progress is exponential, not linear, over time it will become more difficult for the country taking the leap of faith and opting out of competition to recover their lost competitive advantage if other actors do not enter into cooperation. The signal becomes more costly. This makes it less likely any country will leap.
As time progresses and opting out of the race for even just a short time becomes more risky, other countries may be more inclined to not enter into cooperation even after one side takes a leap of faith, because they have more to gain from not cooperating.
As taking a leap of faith becomes more risky, it will also become more difficult for any country to convince internal stakeholders, like voters, that a leap of faith is a sensible action to take.
So all of this goes to say that the best point in time to kickstart serious international cooperation on AI was yesterday, but the second best time is today. And this will be even more true in the future. So let’s leap now!
PS. I haven’t even touched on the fact that a single defector can bring a whole system of carefully built trust and cooperation crashing down. So after international cooperation is kickstarted via a leap of faith, the cooperating parties will have to build a system that strongly disincentivises cooperators from defecting. But this is a topic for another time!