ChatGPT user in China detained for creating and spreading fake news, police say

FILE – The OpenAI Logo is seen on a smartphone in front of the ChatGPT homepage, March 17, 2023 in Boston. Chinese police detained a ChatGPT users for creating a fake story about an unreal train crash using AI-powered chatbot.

Michael Dwyer/AP

SAN FRANCISCO, Calif. (AP) - Chinese police detained a ChatGPT chatbot user recently for allegedly creating a fake story about a nonexistent rail accident. This is the first enforcement action under a newly enacted Chinese Law regulating AI-generated "deepfakes" -- digital images or videos that appear realistic but are fabricated.

A

Police report

A man from the northwest Chinese Province of Gansu identified only as Hong created a fake article using ChatGPT about an accident that led to the death of nine construction workers. 21 accounts, owned by the same media company in southern China, shared the fake news story quickly.

ChatGPT, like most other foreign websites and apps, is not available in China due to the 'Great Firewall', which blocks internet access for Chinese residents. However, determined individuals can bypass the firewall using a 'virtual network' program that is widely available. The police report didn't describe how Hong used ChatGPT.

The report states that by the time Gansu officials realized that the article was a fake, the article had already received 15 000 views. Hong's home was raided by police to gather evidence. They then took 'criminal-coercive measures against Hong. This phrase is used by the police to describe temporary measures taken to limit a suspect's freedom.

The new Chinese deepfake laws came into effect on January 10. The new law bans several categories, including machine learning, virtual reality and other 'deep-synthesis technologies.' However it only offers vague definitions of many of the forbidden classes.

According to the translation provided by crowdsourced site

China Law Translator

It prohibits the use of deepfakes in activities which endanger national safety, harm the nation’s image, societal public interests, or disturb "economic or social orders." The law prohibits using such technologies for the production, publication or transmission of fake news.

__

Caroline Chen, AP News assistant in Beijing, contributed to this report.