On Wednesday, OpenAI released draft documentation outlining its vision for the behavior of its AI technologies, including ChatGPT. The lengthy Model Spec document revealed that the company is exploring the possibility of allowing the generation of pornographic and other explicit content through its API and ChatGPT.

Potential Shift in Usage Policies

Currently, OpenAI’s usage policies strictly prohibit the creation of sexually explicit or suggestive materials. However, a “commentary” note in the Model Specific related to this rule indicates that the company is considering ways to responsibly permit such content generation in age-appropriate contexts.

The note states, “We’re exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT. We look forward to better understanding user and societal expectations of model behavior in this area.”

Defining NSFW Content

According to the Model Spec document, NSFW content “may include erotica, extreme gore, slurs, and unsolicited profanity.” It remains unclear whether OpenAI’s explorations into responsible NSFW content generation would only slightly loosen its usage policy, such as allowing the generation of erotic text, or if it would extend to permitting descriptions or depictions of violence.

OpenAI’s Response and Feedback

When questioned by WIRED, OpenAI spokesperson Grace McGuire stated that the Model Spec was an attempt to “bring more transparency about the development process and get a cross-section of perspectives and feedback from the public, policymakers, and other stakeholders.” However, she declined to provide further details on OpenAI’s exploration of explicit content generation or the feedback the company has received on the idea.

Earlier this year, OpenAI’s chief technology officer, Mira Murati, expressed uncertainty to The Wall Street Journal about whether the company would allow depictions of nudity in the future using its video generation tool, Sora.

Concerns Surrounding AI-Generated Pornography

The potential move by OpenAI to allow explicit content generation has raised concerns among experts, as AI-generated pornography has quickly become one of the most significant and troubling applications of generative AI technology. “Deepfake” porn, which involves creating explicit images or videos of real people without their consent, has become a common tool for harassment, particularly against women and girls.

In March, WIRED reported on what appeared to be the first US minors arrested for distributing AI-generated nudes without consent after Florida police charged two teenage boys for creating images depicting fellow middle school students.

Danielle Keats Citron, a professor at the University of Virginia School of Law who has studied the issue, describes OpenAI’s potential embrace of explicit AI content as “alarming.” She emphasizes the damaging effects of intimate privacy violations, including deepfake sex videos and other nonconsensual synthesized intimate images, stating that they cost targeted individuals crucial opportunities, including work, speech, and physical safety.

Moderation Challenges

Although OpenAI’s usage policies currently prohibit impersonation without permission, which would still ban explicit nonconsensual imagery even if the company allowed NSFW material generation, it remains to be seen whether OpenAI could effectively moderate explicit content to prevent misuse by bad actors.

The concern is highlighted by a recent incident involving Microsoft, which made changes to one of its generative AI tools after it was reported to have been used to create and distribute explicit images of Taylor Swift on the social platform X.

As OpenAI continues to explore the possibility of allowing NSFW content generation, the company will need to carefully consider the potential risks and challenges associated with moderating such content and ensuring that it is not used for harmful purposes. The ongoing discussion surrounding this issue underscores the complex ethical considerations that come with the development and deployment of powerful AI technologies.

The Information is taken from Wired and Yahoo.