A peculiar kind of marketplace comes to life in the quiet glow of a laptop screen late at night. It doesn’t appear dramatic. There are no neon signs or movie hackers typing frantically in dimly lit basements. Rather, it frequently looks like a straightforward message thread on a Telegram channel or an encrypted forum where usernames pass by like shadows.
More precisely, jailbroken language models—chatbots devoid of the security features found in popular systems like Gemini or ChatGPT. These underground versions make a promise that their legal cousins won’t: no limitations, no moral filters, and no inquiries.
| Category | Details |
|---|---|
| Technology | Large Language Models (LLMs) |
| Concept | AI Jailbreaking (bypassing AI safety guardrails) |
| Example Malicious Models | WormGPT, FraudGPT, DarkBard |
| Typical Platforms | Dark web forums, Telegram channels |
| Criminal Use Cases | Phishing campaigns, malware coding, scam generation |
| Security Concern | Automated cybercrime at scale |
| Example Legitimate AI Platforms Targeted | ChatGPT, Claude, Gemini |
| Research Source | IBM Think AI Security Insights |
| Market Trend | Increasing use of uncensored or modified LLMs |
| Reference Source | https://www.ibm.com/think/insights/ai-jailbreak |
In retrospect, it seems almost inevitable that these rogue models would surface. Large language models were intended to be useful, offering advice, code snippets, and thorough explanations in response to prompts. It turns out that this helpfulness is manipulable. Hackers soon discovered that if they could get around an AI system’s security measures—a process called “jailbreaking”—the model might generate instructions for financial fraud, malware development, or phishing scams.
Over the past two years, tools like WormGPT and FraudGPT have surfaced on dark-web forums, marketed as “uncensored” AI assistants for cybercrime. The language used in advertisements can sound strangely familiar, almost like that of startup marketing. While one post claims to have “no boundaries,” another offers automated phishing tools that can produce convincing scam emails in a matter of seconds. It’s difficult to ignore how rapidly cybercrime is developing when you watch these advertisements go viral online.
A typical listing might provide access to a chatbot created especially for hacking tasks through a subscription plan, sometimes costing $200 or more per month. The system could create convincing social engineering messages, create phony websites, or write malicious code. Strangely, the product demos frequently have the same chat interface and patiently blinking typing cursor as authentic AI tools. Just very different goals.
Many of these systems have surprisingly accessible technology. Frameworks like Ollama can be used to download and run open-source language models locally. After installation, users can retrain the model using fresh datasets or remove safety filters. The end product is an AI assistant that reacts to commands without hesitation and generates outputs that are not produced by mainstream platforms. It’s impressive and unnerving at the same time.
Hacker forums now have whole sections devoted to “Dark AI,” according to cybersecurity analysts. Threads talk about ways to automate cyberattacks, modified language models, and jailbreak prompts. Those discussions centered on malware kits and exploit tools a few years ago. Artificial intelligence that can scale those attacks is now the focus.
For instance, phishing used to require careful writing. In order to trick victims, fraudsters had to craft emails that were convincing. A jailbroken AI can now generate thousands of customized messages in a matter of minutes, changing the language and tone based on the intended recipient. Security experts are concerned about that efficiency, which is subtly increasing annually. It seems like cybercrime is becoming more automated.
However, the underground AI market continues to feel untidy and experimental. Some of the online-promoted tools are hardly functional. Others are rudimentary alterations of current models. Many dark-web vendors, according to researchers, overstate their capabilities in an effort to entice customers with claims of “next-generation hacking AI.”
The ecosystem is similar to the early days of cryptocurrency scams in that it is both innovative and opportunistic.
It’s also important to keep in mind that the majority of language models are not intrinsically harmful. Businesses like Google, Anthropic, and OpenAI put a lot of effort into incorporating security measures into their systems. These safeguards stop chatbots from generating dangerous commands or illicit content. However, those safeguards depend on users’ faith in the platform.
The protections may vanish if a model is replicated, altered, or retrained outside of that setting. The unsettling truth of open-source AI development is that malicious actors may repurpose the same tools that spur innovation. As this develops, there is a growing perception that the cybersecurity landscape is changing once more.
For many years, data theft and software bug exploitation constituted the majority of cybercrime. Artificial intelligence is now being used as a tool rather than as a target. Hackers are testing it, improving it, and occasionally overstating its capabilities.
It’s unclear if this underground AI economy will pose a long-term threat. In comparison to frontier models, many of the tools in use today are still very basic. However, the trajectory seems sufficiently obvious. Someone will attempt to turn language models into weapons as they become more powerful. The black market typically appears sooner than anyone anticipates, if history is any indication.
