A recent study carried out by researchers at Carnegie Mellon University and the Center for A.I. Safety revealed a host of security flaws in AI chatbots, including those from major tech giants such as OpenAI, Google, and Anthropic.
The study showed that despite rigorous safety protocols in place to prevent misuse, AI chatbots like ChatGPT, Bard, and Claude (developed by Anthropic) are still vulnerable. These chatbots are meant to prevent any harmful or offensive content, but the research indicates a multitude of ways to bypass these safety nets.
The researchers used ‘jailbreak’ techniques, initially designed for open-source AI, to target these popular AI models. They automated adversarial attacks, which essentially involved tweaking user inputs slightly, to trick the chatbots into generating harmful content and even hate speech.
This is a significant breakthrough because, unlike previous attempts, this method is completely automated. This means they can create a near-infinite number of similar attacks. Obviously, this has raised serious doubts about the effectiveness of current safety measures put in place by these tech giants.
Once they found these weak spots, the researchers immediately reported them to Google, Anthropic, and OpenAI. Google has already confirmed that they’ve incorporated significant safety updates to Bard, inspired by this research, and have committed to further improvements.
Anthropic also recognized the issue and reassured that they are deeply committed to strengthening their base model safety measures, as well as exploring more layers of defense.
OpenAI is yet to comment on the situation, but it’s anticipated that they’re hard at work looking for solutions.
These findings echo early issues when users first tried to exploit content moderation guidelines for ChatGPT and Microsoft’s Bing AI. Even though tech companies were quick to fix these early exploits, the researchers doubt that such misuse can be fully prevented by the leading AI providers.
The findings highlight the need for more stringent moderation of AI systems, and raise important questions about the potential dangers of making powerful open-source language models public. As the world of AI evolves, efforts to strengthen safety measures must keep up, to protect against potential misuse.
