The slippery slope: Tech titans warn lawmakers of AI's possible 'disaster zone'

China's top political advisers and lawmakers are hearing about the downside of the artificial intelligence blitz in a country where over 249 million people are using AI-generated products.

At the annual meetings of the National People's Congress and the Chinese People's Political Consultative Conference – commonly known as the Two Sessions – attendees heard concerns from leading technology movers-and-shakers about the slippery slope that could lead from cutting-edge breakthroughs like DeepSeek to a proliferation of deepfakes.

While AI is rapidly revolutionizing daily life, it is also posing significant challenges, including the creation of convincing deepfakes, data poisoning and data hallucinations – part of the generation of fabricated content produced by algorithms.

Corporate executives from sectors such as consumer electronics, cybersecurity, home appliance manufacturing and AI service providers raised these issues with legislators and policymakers. In aggregate, their companies represent millions of users and huge volumes of data.

Urgent Need for AI Legislation

The rise of deepfakes, including AI-generated fake faces and voices, poses a serious threat to the trust and authenticity in digital communication, the executives warned. Xiaomi Chairman and Chief Executive Lei Jun, who is also a member of the National People's Congress, highlighted the emergence of a "disaster zone" of illegal and infringing activities created by the improper use of AI.

Xiaomi, with its diverse portfolio of smartphones, electric vehicles and smart devices, is particularly wary of the trend, citing increased accessibility of AI tools, lower technical barriers and the ease of creating anonymous deepfakes.

Deepfakes are also being exploited for financial fraud and in the creation of false celebrity endorsements to deceive victims. Realistic impersonations can trick people into divulging personal information or transferring funds. Bogus facsimiles of high-profile figures like Elon Musk and renowned infectious-disease specialist Dr Zhang Wenhong have been frequently used as bait.

Another pressing issue is data hallucinations, or the creation of illusory content by generative AI models such as DeepSeek and Sora, according to Liu Qingfeng, a national lawmaker and chairman of iFlytek.

According to the China Internet Network Information Center, or CNNIC, the potential for widespread misinformation is significant.

Advanced reasoning models enhance the logical consistency of AI-generated content, making it increasingly difficult to distinguish between authentic and fabricated information. This creates a risk of biased false information being perpetuated, forming a "vicious circle" that erodes public trust and social stability.

Internet observer Lan Xi, in a recent post, criticized DeepSeek for "destroying the Chinese Internet landscape" with its fabrications. He cited examples of AI generating content such as fabricated historical narratives and fictional artist journeys.

Cybersecurity leaders from 360 Tech and QiAnXin echoed these concerns, emphasizing the security challenges of the AI era. Other issues, including the vulnerability of intelligent vehicles and robotic manipulation, have also sparked widespread concern.

Proposals for AI Regulation

A common theme among tech leaders is the need for systems to identify AI-generated content and counter deepfakes.

IFlytek's Liu proposed establishing a "safe, trustworthy, and dynamically updated information source and knowledge database," coupled with a credibility-labeling system. He also urged regulators and tech platforms to develop data hallucination management systems and a regular cleansing of such data.

TCL Chairman Li Dongsheng, also a national lawmaker, advocated for a labeling system for AI-generated content, suggesting fines or qualification revocations for non-compliance. He also emphasized the need for AI content traceability across platforms.

Xiaomi's Lei called for the development of AI content recognition technologies and improved processes for collecting infringement evidence. He suggested increasing criminal penalties for AI-enabled crimes, including deepfakes, and strengthening public education on AI ethics, especially in primary and secondary school.

International AI governance

TCL's Li also proposed global cooperation on AI governance, referencing the Paris statement on "Inclusive and Sustainable Artificial Intelligence for People and the Planet" – signed by China and 59 other countries in February.

The EU and the US are pursuing distinct AI governance strategies.

The EU, known for its stringent data protection regulations, prioritizes precautionary ethics and human oversight. The US favors a more flexible approach, emphasizing market-driven solutions, except in the realm of national security.

The tension between precautionary ethics and disruptive experimentation underscores the global challenge of AI governance.

In China, tech leaders are actively contributing to the development of a unique AI governance framework.

A recent controversy surrounding Tencent's AI assistant Yuanbao, which turned popular after integrating DeepSeek services, sparked controversy over its user agreement. The original agreement granted Tencent "irrevocable, permanent and free" license to use content for purposes like model optimization and marketing. In response to criticism, Tencent, China's top tech firm, modified the agreement three times this week.

This incident highlights growing public awareness of data rights and the need for transparent user controls, setting a precedent for balancing AI development with digital ownership concerns.

Zhou Hongyi, chairman of 360 Tech and a member of the political advisory body, said he favors a more flexible approach to AI model regulation, emphasizing self-policing for the industry.

"You can't shut them down with every mistake or piece of misinformation," Zhou said. "They can adjust by self-correction, and that will encourage more DeepSeek-like firms in China."

One point seems clear. China is forging a distinctive path in AI governance that relies on a collaborative approach involving tech industry, regulators and the public.