Without its own version of OpenAI, and lagging in all three key elements for fundamental generative AI development (computational power, quality data and advanced algorithms), China has turned its attention to the vertical model — that is, to AI applications in specific industries such as finance, health care, electric vehicles and driverless cars. China remains well behind in this arena, even as the technological gap narrows.
The release of ChatGPT by OpenAI in 2022 was a watershed moment for AI development. The breakthrough was made possible by the great computational power provided by Nvidia’s advanced AI chips and billions of parameters for model training. China, constrained by a lack of vital computational power, access to vast pools of aggregated data and investment capital, began to fall behind.
Using open-source large language models (LLMs) such as Meta’s Llama to develop its own models, China is now closing the gap. But the country continues to trail by a year or more, which, as former Google CEO Eric Schmidt has said, is an “eternity” in terms of AI.
That’s because China’s LLMs are still being developed on Meta’s open-source Llama model. And the gap between China and the United States in computational power is expected to continue, because of US export restrictions on AI chips. Nor will China’s difficulties in increasing its supply of high-quality data and capital investment ease any time soon.
Among these factors, the shortage of advanced AI chips for fundamental model training is the single most essential element. In effect, this is choking AI development in China and hampering its ability to catch up to the United States. Although Chinese firms such as Tencent, Baidu and Alibaba had stockpiled a quantity of advanced Nvidia AI chips before the US sanctions went into effect in October 2022, and some chips continue to flow into China via the underground market, China still is not getting enough high-end chips to power AI model training.
Meanwhile, OpenAI and other American companies such as Microsoft, Google and Meta continue to quickly advance.
The lack of quality data is another fundamental barrier. There’s inconsistent data collection in most industry sectors and by websites. And the data that is collected is often either unusable or less workable. In addition, China lacks an efficient market for data trading.
On the question of capital investment, the problem is money: LLM training is hugely expensive. Training is energy-intensive and also time-consuming, often taking weeks or months. OpenAI’s CEO Sam Altman has said that training the GPT-4 model cost more than $100 million. Only a few very large companies can afford to do it. Again, this is an advantage for the United States. Data from CB Insights, a business analytics platform for tech market intelligence, shows that in 2023 the United States continued to lead among AI start-ups, accounting for 73 percent of total global AI equity funding, and nearly half of all deals in the sector. By comparison, China in 2023 accounted for just five percent of total global AI equity funding, and 15 percent of all deals.
Another limiting factor for China is its unduly restrictive domestic governance. In July of 2023, Beijing introduced political regulations on generative AI, stipulating that “content generated through the use of generative AI shall reflect the Socialist Core Values.”
In addition, through setting detailed standards to screen input training data and the output of a model, China has greatly restricted the remarkable power of transformers, the machine-learning neural network architecture that laid a foundation for the breakthrough of LLMs and other AI fundamental models.
Industrial applications require fewer but specific parameters, which means they don’t need as many advanced AI chips.
Under these circumstances, Chinese enterprises have little choice but to look for alternative routes for AI development, such as industrial applications and deployment in specific sectors, for instance, personal computers, the Internet of Things, robotics and driverless vehicles.
And although it’s a choice driven by necessity, many industry players seem confident the strategy shows promise. That’s because it entails lower costs and higher cost-performance ratios and plays to China’s competitive advantages: a rich and diversified set of use cases, huge market demand and great industrial capacity. That in turn is built on China’s traditional advantages in manufacturing, such as trained workers with lower labour costs, the scale of its economy, supportive government policies and so on.
All these factors point to this direction for AI development in China: applications in specific industries. These require fewer but specific parameters, which means they don’t need as many advanced AI chips to provide high computer power to train models.
China’s AI Plus initiative, released in its 2024 government work report, strongly reinforces this thrust, aiming to further integrate AI and various industries in the real economy. And industry is following Beijing’s lead. Tech giants in China such as Baidu, Alibaba, Tencent and Huawei have focused on developing highly specific AI applications in areas such as autonomous driving, biology, weather forecasting, education, mining and the like.
However, great concerns remain among the Chinese high-tech industrial community as to whether China’s alternative approach will succeed.
Indeed, a widening gap with the United States in fundamental model development could derail the strategy. Continuously improving fundamental AI models such as GPT-4, or multimodal tools such as Sora, may soon be capable of managing all key functions in specific sectors.
Simply put, improved fundamental AI models could kill most of the start-ups that have developed industry-specific AI tools, based on the large, general models. As OpenAI’s CEO Sam Altman has said, continuous improvement of AI models could eliminate the need for additional features, as the large models become more versatile.
Looking ahead, the United States is likely to retain its lead in this sphere. Given restricted availability of AI chips, quality data and algorithms, the best scenarios for China’s big companies such as Baidu, Alibaba, Tencent, Huawei and ByteDance would be to continue to work to gain ground in fundamental model development, while focusing on specific applications.
Chinese companies have a history of innovation built on US technological breakthroughs, as in the era of mobile internet in the early 2010s. In that instance, China caught up with and even surpassed the United States in applications such as e-commerce, mobile payments, and early voice and facial recognition.
This time, again, instead of seeking technological breakthroughs and leading revolutionary innovation, Chinese companies are focusing on use cases and applications that help consumers or businesses to make quick money. The approach underlies the ongoing chaotic phenomenon in China’s fundamental model market, in which more than 100 LLMs emerged in a single year following ChatGPT’s release.
Most of these models are basically fine-tuned versions of LLMs that duplicate open-source models such as Llama or are patchworks based on different models that are seeking to quickly profit in China’s fledging LLM market.
There are only a few developed by leading companies, such as Baidu’s Ernie 3.0 (Wenxin Yiyan) and Alibaba’s Tongyi Qianwen, and start-ups such as Baichuan, Moonshot AI, Zhipu AI and MiniMax are trying hard to get advanced AI chips and billions or even trillions of parameters with which to train their models.
There is a possibility, as some Chinese observers have claimed, that the widespread industrial applications of AI in the huge number of use cases could help China repeat its history of innovation built on US technological breakthroughs and create more high-quality business-specific data, This, in turn, would provide more application scenarios and thus create a real industrial revolutionary moment.
For example, AI’s widespread use in disease diagnosis and treatment in medical scenarios in China could create even more quality data to help the development of the AI solutions market and bring sweeping changes in future health care. Industry verticals in a variety of use cases, once integrated with fundamental models, could provide a well-established base for China to catch up or even lead in AI-driven industrial transformation in the future.
The debate continues. Will the constant improvement of fundamental models by big companies in the United States replace most other vertical models in specific industries? If the answer is yes, China’s alternative route for AI development could be a dead end. But if the answer is no, there may be an opportunity for the country to reap the harvest of AI development. The final answer, as yet unknown, may fall between the two.
