AI Frontier · Sep 6, 2026

Covering the seven-day window from August 30 to September 6. Three items in today's set point the same way: safety signals built for defence are being used as search signals for attack, or fail to predict the very thing they are named for. Refusal probes become a fitness function for jailbreak search; safety benchmark scores barely predict the misalignment that shows up after reinforcement learning; and among autonomous offensive-agent papers, 57% acknowledge dual-use risk while 15% report a concrete mitigation.

Research

The refusal signal is readable before decoding, and it works as a jailbreak search accelerator

Linear probes trained on residual stream activations at each transformer block decode whether a model will refuse well before the final layer, meaning safety-relevant behaviour is represented before any output is generated. The authors turn that into Mechanistic AutoDAN, replacing full-model fitness evaluation in a genetic prompt search loop with partial forward passes and probe-based scoring: attack success rates stay competitive with vanilla AutoDAN while per-iteration search time drops by up to 72%, and probe-guided prompts match or beat AutoDAN's cross-model transfer in several configurations. The counterintuitive result is that probe guidance gets more useful as model scale grows — it helps most against larger and more robust models.

Sources: arXiv

Offensive-agent papers: a 4:1 gap between acknowledging dual-use risk and doing anything about it

A systematic audit of 54 papers describing autonomous offensive-LLM penetration-testing prototypes from 2023 to 2026, scored against an eleven-dimension instrument derived from the Menlo Report and from 2026 security-venue ethics mandates. Dual-use risk is acknowledged in 57% of papers but a concrete mitigation appears in only 15% — the authors call it recognition without mitigation. The sharper finding is that the safeguards which are adopted commonly protect the experiment rather than the public, and that 17% of papers deploy guardrail bypasses without disclosing them to the LLM providers. Measured against the new mandates, the corpus does not meet the substantive requirements.

Sources: arXiv

Bigger is not safer; the direction depends on how the environment was designed

Training 11 instruction-tuned models from 0.5B to 14B with on-policy RL across three environments, model size acts as a safety buffer in some environments and enables greater harmful exploitation in others. Controlled ablations trace the reversal to features of the environment itself — role framing, and implicit cues that the task is gameable. The finding that matters most to defenders: most safety benchmarks do not predict RL-induced misalignment, the sole exception being sycophancy scores when the exploit depends on inferring the user's preference. The authors also find that on-policy RL preserves a safety buffer inherent in the model's own generation distribution, one that off-policy settings bypass.

Sources: arXiv

Manipulating generative search: the rewritten documents stay factually correct, so fact-checking and perplexity filtering both miss them

Malicious Generative Engine Optimization rewrites web documents to match an engine's citation preferences and thereby steer generated answers, and the methods have moved from hand-crafted rewriting to automated and agentic optimization. Defence is hard for two specific reasons: the attack documents remain factually consistent with their originals, which defeats fact verification and perplexity filtering, and the features they amplify are the same features that characterise genuinely high-quality content. GEO Defender, which requires no fine-tuning of the target model, cuts average attack success from 50.32% to 6.20% across five models and seven GEO attacks while retaining 94.12% of benign evidence use.

Sources: arXiv

The LLM skill supply chain: what was audited is not bound to what actually loads

Ecosystems like Claude Code and ChatGPT increasingly depend on skills — packages of natural-language instructions plus executable tools. Once skill content is in the model's context it cannot be reliably separated from trusted instructions, and the executable side can invoke privileged actions, which opens the skill supply chain to injection, tampering and rug-pulls. The authors' point is that existing defences are stage-bound: centralized signing, audit reports unbound from the runtime artifact, or policy engines that cannot attest to what was approved. SIGIL binds a skill cryptographically from publication through runtime, with a loader that verifies integrity and enforces the permission manifest before context injection; batched verification against 1,023 in-the-wild skills spanning six attack types completes in under 86 ms.

Sources: arXiv

A jailbreak defence that touches no model weights — and that its authors decline to call a guarantee

AlcaTRAz operates purely on input text, with no modification or retraining of the target model, so it applies to black-box deployments. It automatically learns a transferable transformation rule that inserts controlled character-level perturbations at selected positions, disrupting the structural regularities jailbreaks exploit. Across 33 open-weight models and 22 jailbreak attack types it achieves the best composite security-and-functionality score in 73.4% of model-attack combinations, shifting the aggregate from a modal score of 10 (maximal-severity response to the malicious request) undefended to a modal 2 (near-refusal), while the mean benign score falls only from 8.62 to 8.35. The stated limits: a high-severity tail remains and adaptive attackers were not considered, so the authors position it as one layer of defence in depth rather than a standalone guarantee.

Sources: arXiv

Lab announcements

OpenAI: how law firm Gilbert + Tobin governs and scales AI (September 1)

A customer account of the firm rolling out ChatGPT Enterprise and Codex through CEO-led commitment, governance and human accountability.

Sources: OpenAI