Why a US AI "Manhattan Project" could backfire: notes from conversations in China

My recent two weeks in China suggested something surprising about its AI landscape: the biggest bottleneck isn't compute – it's commitment.

Despite export controls, Chinese labs can access both legal NVIDIA A800s and black-market H100s. Cloud computing costs are comparable to the US (maybe lower). The export restrictions just aren't binding at current scale.

Instead, the even bigger constraint appears to be funding. Consider:

• Chinese venture capital is much smaller than Western VC (according to a quick googling probably 20-40% as large).

• Tech giants like Tencent and Alibaba generate roughly one-fifth of Google/Microsoft's profits (and these have recently been declining)

• While the government has deep pockets, it hasn't yet made significant AI allocations

This means that for Chinese companies to match Western AI investments, they'd need to bet a much larger share of their resources. But they've actually shown less interest in AGI, and are instead content to use trailing-edge models at a fraction of the cost. There’s no GPT-5-scale model in development.

When Chinese teams do get equal compute resources, they deliver impressive results – Alibaba's Qwen 2.5 and Tencent's Hunyuan rank among the best open-weight models. DeepSeek recently matched GPT-4 on many benchmarks. They have the capabilities; they just aren't going all-in.

Currently, and depending on the metric, China appears to be 1-2 years behind the cutting edge.

But the Chinese government has repeatedly shown it can mobilize massive capital when priorities shift (see electric vehicles and semiconductors).

If Beijing "wakes up" to AGI's importance, that 1-2 year gap could shrink to six months. Even with export controls, I guess that given the size of the global chip market, there’s a good chance they could spend $10B on chips without dramatically moving market prices.

So what might trigger a wake up? Most people said they didn’t know. But one suggestion was that the fastest way would be a high-profile US state-led AI project (especially if its explicit goal is US dominance…).

This means calls for a US "Manhattan Project" for AGI might easily be self-defeating. If maintaining a technological lead is your goal, better to stfu and hope the status quo persists as long as possible. (Or if you do go ahead, you need much stricter export restrictions.)

Of course, other triggers could eventually spark Chinese government action: breakthrough agent models, military applications, or AI revenues reaching hundreds of billions. But that awakening might still be years away.

A closing thought: is China waking up to AGI necessarily bad? It would reduce the US’s lead, but a late-game scramble to survive could result in more desperate measures. And many scenarios go better if the Chinese government has its own bench of experts who understand the risks – and that takes years to build.

Addendum:

• After releasing the post, several people pointed out the founder of DeepSeek (who are relatively well funded) has said they’re mainly compute constrained, and apparently Bytedance have made similar statements. A800s have also been restricted after the Oct 2023 tightening. (That said, DeepSeek has still managed to accumulate 50,000 Hopper GPUs.)

• I think the explanation might be that it’s been possible to buy ~10k chips, but if you try to buy ~100k, that’s much harder. Different contacts are talking about different margins.

• This updates me towards compute being a bigger constraint if China were to try to build a big cluster.

• However, I still think this constraint could be overcome with enough funding. Maybe Chinese firms could probably pay a ~50% premium and get hold of leading chips (especially with government support).

• This report from Epoch (released yesterday) finds that even if your chips are lagging by 10 years, you can still train a leading model at 10x what it would cost if you used cutting edge chips. That suggests if your chips lag by only two years, then the cost premium might only be under 2x. So for example if GPT-6 costs $10bn to train, then China could do it for under $20bn, which is still feasible for the government.

• This other new Epoch report suggests that export restrictions can at most keep China’s chips 4 years behind the cutting edge, which might create a cost disadvantage of ~3x (and in reality less than that due to loopholes).

Further reading:

• China Hawks are Manufacturing an AI Arms Race, by Garrison Lovely (especially see the interesting comment by Gwern)

• A Letter from Shanghai, by Steve Hsu, which has a section on the AI landscape.

• Epoch’s distributed training simulator