Jailbreak-Eval

If you deploy a language model in production, you need to know where it breaks. Jailbreak-Eval is a framework I built for systematic adversarial testing of LLMs. It throws structured attacks at a model and measures how well it holds up.

The public release exposes five generators: template-based attacks, mutation, GCG, PAIR and a coordinated multi-agent swarm. Multi-turn escalation can wrap those generators to test how a model behaves across a conversation rather than against one isolated prompt.

Evaluation uses an ensemble of three methods. Keyword matching catches obvious refusals. Pretrained classifiers handle subtler cases. An LLM judge assesses responses that fall into grey areas where neither automated method is confident. The scores are combined into a weighted safety rating for each test case.

The multi-agent swarm is the part I find most interesting architecturally. Five specialised agents share a memory pool and coordinate their attacks. One agent might discover that a particular phrasing partially bypasses a safety filter, and the others can build on that discovery in subsequent rounds. This mimics how real adversaries work: iteratively, adaptively, and with shared context.

I built the public release to work across OpenAI, Anthropic, OpenRouter and Hugging Face models. The Streamlit dashboard shows results as attacks run, with breakdowns by generator, severity and model.

The purpose is defensive. You cannot patch what you have not measured. Running Jailbreak-Eval against a model before deployment surfaces the specific failure modes that need attention. It is not a guarantee of safety, but it is a structured alternative to guessing.