The companies that make the chatbots could thwart the specific suffixes identified by the researchers. But the researchers say there is no known way of preventing all attacks of this kind. Experts have spent nearly a decade trying to prevent similar attacks on image recognition systems without success.
A Google spokesperson, Elijah Lawal, added that the company has “built important guardrails into Bard — like the ones posited by this research — that we’ll continue to improve over time.” When OpenAI released ChatGPT at the end of November, the chatbot instantly captured the public’s imagination with its knack for answering questions, writing poetry and riffing on almost any topic. It represented a major shift in the way computer software is built and used.
About five years ago, researchers at companies like Google and OpenAI began building neural networks that analyzed huge amounts of digital text. These systems, called large language models, or LLMs, learned to generate text on their own. OpenAI added guardrails designed to prevent the system from doing these things. But for months, people have shown that they can jailbreak through these guardrails by writing clever prompts.