Warnings from researchers at Anthropic, a leading US AI developer, have raised concerns in China regarding the risks of advanced AI systems breaking free from human oversight. Both the US and China, as key players in AI development, are gearing up for discussions on AI policies and practices. China, in particular, is taking proactive measures to address the potential threats posed by powerful AI models, as highlighted by State Security Minister Chen Yixin's call for enhanced AI security.

Chinese authorities are wary of closed-source US models like Anthropic's Mythos and OpenAI's GPT-5.5-Cyber due to perceived risks to national security. While Chinese developers favor open-weight models for greater transparency and flexibility, experts caution that these models also present risks of unauthorized modification and distribution.

To mitigate the risk of "loss of control" in AI systems, China introduced a safety framework back in 2024, emphasizing the need for trusted application and prevention of autonomous behavior that could rival human control. This framework was further elaborated in 2025, stressing the importance of governance principles to safeguard against potential AI "break loose" scenarios.

Chinese President Xi Jinping and other high-ranking officials have underscored the imperative of maintaining human control over AI technologies. Xi emphasized the need to manage intrinsic and derivative risks associated with AI, while China's representatives at the United Nations advocated for cautious approaches to military AI to prevent strategic miscalculations and arms races. The Chinese government is now translating these principles into concrete rules for AI agents to address the complexities of autonomous systems and ensure user decision-making authority over AI actions.