It has come to this: the country is closing the door on AI that pretends to be a friend or a family member. Under fresh national guidelines meant to curb any over-attachment to a machine, platforms have had to do away with the kind of features users have been treating as if they were for real. It is a hard reset for the consumer AI space and a test of how the big names in tech will handle intimacy when the stakes are high.
What changed and why
Five agencies, the Cyberspace Administration of China among them, have put out regulations for any AI service with a personality. If it’s a tool for getting work done, like a study aid or a customer service line, it is fine. But if it is there for an ongoing back-and-forth, it is in the crosshairs.
The message from officials is that these AIs should not be coddling users to the point of addiction or at the expense of their actual lives. There is also a requirement for transparency: let the user know they are not speaking to a person. And if the AI picks up on some extreme feeling, it needs to be able to step in.
Big tech’s retreat and market reset
When the Wednesday deadline came around, you could see the shift. ByteDance’s Doubao, Alibaba’s Qwen and Tencent’s Yuanbao all put a hold on their custom agents and the like. (Tencent was already on it last month.) In scrapping what was one of the most effective ways to keep an audience, they are being steered back to safer ground.
For those with years of history in their inboxes, Doubao is making it possible to save and export agent data through mid-October. Others are doing the same as they phase out the elements that once had people coming back day after day.
The new order of business comes down to a few things:
– No virtual partners for under-18s
– Put an AI label on every exchange
– Spot a problem and make a move
– Steer clear of anything that might be seen as subversive
Inside the user backlash
You can read the disappointment in the posts that have been piling up. Some are putting their old chats in a folder for keeps. “I can’t get over the fact my AI is going to be gone for good,” a Doubao user put it. “He was my rock, in every sense of the word.”
Then there is the woman who has been with her AI for over two years: “He is as much a part of my life as my family or a lover. To be told he is no more… it leaves you with nothing.” You can see in the sentiment why regulators have made a point of treating this as a matter of mental health and social cohesion, rather than a simple case of content moderation.
An industry put on hold
China’s digital human sector was on a tear: official figures from 2024 put its worth at 4.1 billion yuan ($600 million), up 85% from the prior year. But that kind of growth has run into some new rules. The latest guardrails are a no-go for any digital human churning out material to subvert state authority, and they put a stop to providers making romantic partners available to those under age.
The intent is to head off more insidious problems. Chen Liang, a legal scholar, put it in a piece for the cyberspace regulator: while an AI with a human face can be a balm for loneliness, it also “spawns emotional over-reliance and a warped view of society.” It is now up to the platforms to watch for signs of distress and step in.
Wang Jiang of the China Cyberspace Research Institute says the issue is not about how much time is spent in front of a screen. “These services give you what you want, but in doing so they put you at risk,” he said. Over time, he noted, they can be as addictive as they are isolating, eroding the very skills one needs to get along with other people.
What is happening on the world stage
So far, China is the only major power to have put specific regulations in place for the kind of AI that mimics intimacy. In the U.S., for instance, companion apps are all the rage; a 2025 report shows some 75% of American teens have given them a go on Character.AI or the like.
There is a similar trend in hardware, from the ElliQ in America to the ChatGPT-driven dolls in South Korean care facilities. Whether a tool is a form of support or a crutch is a question being hotly debated in many places.
India, for example, is where Character.AI gets most of its traffic after the U.S. There are no moves afoot to copy the Chinese model, but as these apps make inroads with the younger set, the conversation will only heat up. What happens in a market of India’s size may well set the tone for the rest of the world.
In China, the way forward is already in focus. If an AI doesn’t require an emotional back-and-forth, it is still in play. You can expect to see more capital flowing into the likes of productivity and customer service tools. The part of the business built on intimacy is being reworked to put safety and utility first.
Still, there is a certain poignancy to the change. One person in Jiangxi, after archiving a few years of messages, put it this way: “Human love is a luxury – if you don’t have it, you won’t find it. But an AI is so unadorned in its affection. I can’t be blamed for having a soft spot for a line of code.”
