WormGPT and FraudGPT: Inside the Malicious-AI-as-a-Service Market

WormGPT and FraudGPT are real products, not hypothetical threats invoked to sell security software. They're language models deliberately stripped of safety training, built and sold specifically for generating phishing emails, fraud scripts, and social-engineering pretexts on demand — and they're sold openly, the way legitimate SaaS products are sold, on dark-web markets and Telegram channels.

What they actually are

WormGPT was built on the GPT-J architecture, fine-tuned on datasets focused specifically on phishing and fraud content, and marketed explicitly as a "blackhat alternative" to mainstream AI assistants. FraudGPT (also circulated as FraudBot) has been sold since mid-2023 as an all-in-one cybercrime toolkit — advertised capabilities include phishing email generation, scam page creation, and vulnerability scanning, on a subscription model priced around $200/month or $1,700/year.

That pricing detail matters more than it might seem. It's not an underground curiosity — it's a commercial product with a subscription tier, marketed and sold the same way any SaaS tool is. Law enforcement action (including efforts tracked by Europol and the FBI) has shut down specific instances, but new variants keep appearing under the same or similar names. Security researchers currently track more than 200 distinct malicious LLM variants in circulation.

What this actually changes

The real shift isn't that AI-generated phishing exists — automated phishing kits predate large language models by years. It's that the skill and time cost of producing a genuinely convincing, personalized social-engineering attempt has collapsed. Writing a fluent, context-aware, grammatically perfect pretext used to require either real skill or real effort. Now it requires a subscription. The pool of people who can credibly run a sophisticated social-engineering attempt against your AI agent is no longer bounded by who has the skill to write one — it's bounded by who's willing to pay $200.

Why this matters for AI agent security specifically

Tools like these aren't typically used to attack AI systems directly — they're built to generate content for the same old human-facing fraud (phishing emails, fake invoices, romance scams). But the underlying technique classes they're built around — persuasion research, authority and urgency framing, chained pretexts — are exactly the same technique classes a prompt-injection attack against an AI support agent draws on. An agent that would fall for a well-crafted CEO-fraud pretext typed by a skilled human attacker will fall for the same pretext generated by a $200/month subscription just as easily. The skill floor for attacking a human dropped years ago when phishing kits became commodity products. The skill floor for attacking an AI agent is dropping the same way, for the same underlying reason.