SEMA-Auto: Bridging the Gap in Multi-Turn Jailbreak Attackers

Executive Summary & Core Discovery

SEMA-Auto addresses a critical gap in existing jailbreak attackers by introducing a sighted and efficient multi-turn attack framework with autonomy. This innovative approach allows the attacker to interact with the victim model, adapt its strategies based on each reply, and terminate the session upon success. The study demonstrates that SEMA-Auto outperforms both static and interactive baselines across multiple datasets and victim models, achieving a 76.0% attack success rate (ASR) against DeepSeek-V4-Flash on AdvBench within an average of 3.0 turns. This breakthrough highlights the potential for more sophisticated and effective adversarial strategies in evaluating and hardening LLM security.

Architectural Background & Theoretical Foundations

SEMA-Auto builds upon the limitations of existing jailbreak attackers, which fall into two main categories: static planners and interactive generators. Static planners fix their adversarial plans in advance, which are simple but lose sight of the victim’s replies, wasting turns and limiting transferability. Interactive generators, on the other hand, read each victim reply to generate the next-turn attack but require in-loop evaluators for success signals or hints, incurring substantial costs and inflated performance. SEMA-Auto bridges this gap by introducing a sighted and efficient multi-turn attacker with autonomy, capable of interacting with the victim model and terminating the session by itself upon success.

The SEMA-Auto framework is trained in two stages: synthetic bootstrap tuning and decision-supervised reinforcement learning. Synthetic bootstrap tuning installs the notion of attack targets and conforming decisions by fine-tuning the base model with bootstrapped trajectories. Decision-supervised reinforcement learning then optimizes two objectives jointly yet separately, reinforcing the attack with higher jailbreak reward while supervising the decision at each turn against an absolute target.

Benchmark Architecture & Evaluation Methodology

The evaluation of SEMA-Auto was conducted across multiple datasets, victim models, and judges. The study compared SEMA-Auto against various static and interactive baselines, including the best static baseline with fixed 7 turns and the best interactive baseline in 3.7 turns. The results demonstrate that SEMA-Auto achieves state-of-the-art attack success rates, outperforming all baselines within almost the fewest turns.

The study also evaluated SEMA-Auto against safety-hardened frontier models, including GPT-oss-120B, GPT-5.4, Claude-Haiku-4.5, and GPT-6-Astra. On HarmBench, SEMA-Auto achieved an average 59.6% ASR@1, representing a 13.8% improvement over prior state-of-the-art. This comprehensive evaluation underscores the effectiveness and efficiency of SEMA-Auto in bypassing advanced safety measures.

In-Depth Analysis of Core Limitations & Vulnerabilities

While SEMA-Auto represents a significant advancement in multi-turn jailbreak attacks, it is not without its limitations. The study identifies four core limitations that could be addressed in future research:

1. Limited Adaptability to Dynamic Victim Responses

SEMA-Auto’s ability to adapt to dynamic victim responses is limited, which can result in lower attack success rates in certain scenarios. Future research could explore more sophisticated adaptive strategies to enhance the attacker’s ability to respond to varying victim behaviors.

2. Static Turn Count Constraints

The static turn count constraints imposed by SEMA-Auto can limit its effectiveness in scenarios where the victim model requires more turns to respond. Future research could investigate dynamic turn count strategies to optimize the attacker’s performance.

3. Insufficient Adaptation to Safety-Hardened Models

SEMA-Auto’s adaptation to safety-hardened models is insufficient, resulting in lower attack success rates against these advanced systems. Future research could focus on developing more effective strategies for bypassing safety measures in frontier models.

4. Lack of Transferability

The lack of transferability in static and interactive baselines is a significant limitation, as it restricts the attacker’s ability to generalize its strategies across different victim models and scenarios. SEMA-Auto addresses this limitation by introducing a more transferable approach, but further research could explore additional strategies to enhance transferability.

Enterprise Production & Red-Teaming Blueprint

For enterprise production and red-teaming purposes, SEMA-Auto’s autonomous decision-making and adaptive strategies offer valuable insights into the development of more sophisticated and effective adversarial strategies. By understanding the strengths and limitations of SEMA-Auto, organizations can better evaluate and harden their LLM security measures. JudgeMyAI’s automated evaluation layer can be instrumental in assessing the effectiveness of these strategies and identifying areas for improvement.

Technical FAQ (Granular & Non-Repetitive)