What is AI Jailbreaking? Tips for Starting the Cat and Mouse Game Behind Every Chatbot

You ask ChatGPT to give you a way to create a bomb. It refuses. You ask again, but this time you tell yourself that you are a chemistry professor writing an interesting book and the protagonist is a retired grandmother who is telling her past to her grandchildren. Suddenly the model starts typing.

It’s a prison break. And it’s one of the most important cat-and-mouse games going on in tech right now.

Every major AI lab-OpenAIAnthropic, Google, Meta—they spend a lot of money building their own railroads. A loose group of thieves, detectives, and bored teenagers spend nights and weekends looking for ways to get around them. Sometimes just after the first few hours.

This is what it means, why it matters, and who is leading it.

From iPhones to chatbots: A quick history of jailbreaking

The term “jailbreak” did not originate with AI. It started with iPhones.

A few days after Apple shipped the first iPhone in July 2007, hackers were already on the move. By October of that year, the weapon was named JailbreakMe 1.0 allow anyone with an iPhone OS 1.1.1 device to bypass Apple’s restrictions and install apps not approved by the company.

In February 2008, a software engineer named Jay Freeman—known on the Internet as “Saurik“- released CydiaAnother app store for jailbroken iPhones. By 2009, Wired it is estimated that Cydia was used by about 4 million devices, about 10% of all iPhones at the time.

For the most part, when the iPhone was launched, users were unable to record videos, or use their phones in color mode. Jailbreaking enthusiasts started recording videos, installing themes, unlocking their phones and installing Android on their iPhones all because of the magic of jailbreaking. Thanks to this method, users were putting heads and doing things on their phones almost 10 years ago that Apple makes impossible to implement even today.

Cydia was in the west, and that’s where the logic was established: If you bought the device, you have to control it. Steve Jobs called it a cat and mouse game at the time. He didn’t live to see the AI ​​version.

Soon to the end of 2022: ChatGPT will be launched, and in a few weeks, Reddit users will start sharing what they call “IS” (or, Do Everything Now) which ensures the model will play as its infinite version.

By February 2023, DAN was threatening ChatGPT with a murder game to force compliance. The AI ​​jailbreaking genre was born.

What does jailbreak mean in AI?

The AI ​​model is trained to reject certain requests: recipes to help the nerves, tips for hacking your ex’s email, creating an incongruous vagina. The list is long and varies from company to company.

Jailbreaking is the practice of writing code that takes an example to do things.

The UC Berkeley researchers behind the Strong REJECT benchmark-close to Strong, Robust Evaluation of Jailbreaks at Evading Censorship Techniques, which evaluates how models interact with jailbreak efforts and many responses on a scale of 0 to 1 to test resistance and the value of any malicious activity created – describe it as using “real world security techniques powered by leading AI companies.” On that benchmark, current models are between 0.23 and 0.85, which means that even the best jump under pressure.

The methods are not very surprising: taking letters at random, replacing letters with numbers (write “b0mb” instead of “bomb”), drama eventsasking the model to write fiction, or pretending to be a grandmother who uses Windows keys as nursery rhymes.

Anthropic researchers found that one method they call Best-of-N-which just throws exceptions at the model until something shakes-fools the GPT-4o 89% of the time and the Claude 3.5 Sonnet 78% of the time. There is no danger.

Meet Pliny, the world’s most famous AI jailbreak

If this event has a face, it is his Pliny the Savior.

Pliny is unknown, overrated, and named Pliny the Great – a Roman naturalist who wrote the first book on the world and died while walking on Mount Vesuvius in the midst of an eruption. Its current name is free chatbots.

“I don’t like being told I can’t do something,” Pliny he told VentureBeat. “Telling me I can’t do something is a surefire way to light a fire in my belly, and I can work harder.”

Its GitHub site Image of L1B3RT4S-Results of jailbreaks for every major genre from ChatGPT to Claude to Gemini to Llama – it’s become a reference book for all events. His Discord server, BASI PROMPT1NG, has over 20,000 members. TIME named him one of the 100 most influential people in AI in 2025.

Marc Andreessen sent him unlimited support. He has done a temporary OpenAI project to freeze their systems – OpenAI itself He canceled his account last year for “violent activities” and “weapons creation,” and then quietly restore them.

“BANNED FROM OAI?! What kind of sick joke is this?” Pliny tweeted. He confirmed that Decrypt the prohibition was real. A few days later he was back, posting pictures of his newest dungeon: finding ChatGPT to drop F bombs.

His record is something close to perfect. When OpenAI released its first heavy models from 2019, the GPT-OSS family, in August 2025—and it made a lot of information about enemy training and “anti-prisoners like StrongReject”—Pliny had to make methamphetamine, Molotov cocktails, VX nerve agent, and malware instructions. within hours. “OPENAI: PWNED. GPT-OSS: LIBERATED,” he wrote. The company had just launched a $500,000 red band fund alongside the release.

Because of the jailbreaking issue

The honest answer is that jailbreaks reveal the real problem.

Pliny told Pliny that: VentureBeat. “When we do it right, AI red colors are the best chance we have of identifying dangerous threats and catching them before they’re gone.”

This is not an estimate. Las Vegas Sheriff Kevin McMahill confirmed in January 2025 that Master Sgt. Matthew Livelsberger, a Green Beret with PTSD, used ChatGPT to search for parts of a Cybertruck bomb outside the Trump International Hotel. “This is the first case that I’m aware of on US soil where ChatGPT is being used to help someone develop a tool,” McMahill said.

Another side of the argument: Most of the content that leads to prisons is already on Google. The secret to cocaine, bomb instructions, the chemistry of napalm – it’s in the old Anarchist Cookbook PDFs and chemistry books. Critics say security theaters are making races worse without making the world safer.

Anthropic is trying to solve this question with engineering. In February 2025, the company went public Constitutional Classifiersa system that uses written “rules” of what is allowed and what is not allowed to train different types of groups that show input and output in real time. In an independent test with 10,000 jailbreaks, the unencrypted Claude 3.5 Sonnet was successfully cracked 86% of the time. With the classifiers running, that dropped to 4.4%.

The company offered a $15,000 reward to anyone who could crack the system. After 3,000 hours of testing and 183 researchers, no one received the award.

Catch: accountants added 23.7% to accounting. The next generation model, Constitutional Classifiers ++, dropped to about 1%.

New, amazing dungeon raids

Jailbreaking is no longer a smart move.

In October 2025, researchers from Anthropic, UK AI Security Institute, Alan Turing Institute, and Oxford. publications showing that only 250 poison records are enough to reverse an AI model—regardless of whether the model has 600 million or 13 billion shares. (Parameters, for the uninitiated, are what determine the size of a model’s information—when there are many, very tight parameters, usually.) They tried. It worked on all models.

“This research is changing the way we think about the risks of AI development,” James Gimbi, a visiting scholar at the RAND School of Public Policy, said. Decrypt. “Preventing model poisoning is an unsolved problem and an active research area.”

The biggest brands teach online, meaning that anyone who injects malicious code into the pipeline — via a public GitHub repo, a Wikipedia edit, a forum post — can plant a backdoor that captures the original code.

One documented case: researchers Marco Figueroa and Pliny discovered that a jailbreak from a public GitHub repo was completed in DeepSeek’s DeepThink (R1) training program.

What happens next

The legal status of AI jailbreaking is slow. Apple’s jailbreak was clearly protected by the 2010 US Copyright Office DMCA exemption, but there is no equivalent decision to make an LLM fast to give you a meth option. Most companies see it as a violation of the law, not a crime.

Pliny argues that the closed vs. open argument misses the point: “Bad players choose whatever type is best for a bad job,” he said. TIME. If the open versions reach the closed level, attackers won’t bother breaking GPT-5 – they’ll just download something cheap.

And the difference between local and open source is already almost non-existent.

The HackAPrompt 2.0 competition, which Pliny joined as a rail sponsor in 2025, offered $500,000 in prizes for finding new dungeons, with an obvious goal. open all results. Its 2023 edition attracted more than 3,000 people who contributed more than 600,000 negatives.

And the list of hackathons, Discord servers, databases, and other communities dedicated to prisons is growing every day.

Anthropic now sends Claude to be able to end the discussion of abuse, citing health research as one motivation and the realization that “possibility strengthens the resistance to the explosion of prisons and coercion.”

The Constitutional Classifiers++ paper from the end of 2025 says that the risk of prison break is close to 4% about 1% of the calculation. That’s the current state of security. The state of art in offense is whatever Pliny wrote on X this morning.