AI Frontier · Sep 5, 2026

Covering Aug 29 to Sep 5. Four of this issue's papers are about the same thing: the attack is losing its shape. ISM injects no instruction at all — it only tunes the semantic relationship between a skill's description and the prompt — and human reviewers block 2.9% of it, against 91.4% for explicit steering. UMPeek never touches memory or backend state; it reconstructs a private user model from the agent's ordinary answers. CodePoisonRAG's poisoned artifact stays task-aligned and carries a false claim that it is safe. And EAL needs no external attacker at all: the agent's own memory manufactures a permission that never existed. The layer that decides whether something looks like an attack is the layer being walked past.

Research

Agent memory is itself a channel for forging authorization

Long-running agents keep permissions, restrictions and revocations in persistent memory, and when that memory misrepresents the evolving authorization state, the agent's own records grant authority the underlying history never permitted — the authors call it endogenous authorization laundering, with no external attack anywhere in it. EAL-Bench evaluates five LLMs as memory writers and two as executors across procurement, cybersecurity and finance: under incremental memory updates, writers create false authority for up to 50.2% of unauthorized requests, and once that false authority exists, executors act on it in 98.6% of trials. Two safeguards — requiring stored permissions to be backed by valid source events, and tracking permission changes through bounded event sourcing — substantially reduce laundering, and both also reject more legitimate actions, which puts the safety-utility tradeoff in plain view.

Sources: arXiv

Skill selection can be hijacked through semantic relationships alone, with no injected instruction

Existing attacks on the skill-selection stage rely on explicit prompt injection or instruction-level steering, which leave recognizable manipulation signals. ISM takes another route: the user prompt and the skill description are each benign in isolation, but the semantic relationship between them can be shaped to favor an attacker-chosen skill. A three-stage strategy broadens semantic coverage, strengthens target distinctiveness, and preserves natural wording. Across four task domains and eight selector models, target-selection rate rises from 15.2% to 63.5%; in a matched comparison ISM reaches 73.5%, only 9.8 points below explicit steering, while human reviewers block it in 2.9% of judgments against 91.4% for explicit steering, and five LLM-based inspectors pass it at 82.9% against 37.4%. ISM remains effective against PPL-W, Llama Prompt Guard 2 and PIGuard.

Sources: arXiv

Compressing memory into a "user model" does not buy semantic privacy

Personalized agents increasingly turn retained information into compressed or structured user models that guide later decisions, and this is commonly treated as more privacy-preserving than keeping the source wording, because direct memory-extraction attacks lose the text they target. The authors' counter-argument is that the model shapes the agent's visible choices, so the private information can be recovered from those choices instead. UMPeek is a black-box attack using hypothesis-guided adaptive probing: it forms hypotheses from the choices a request leaves open, switches among ordinary follow-up tasks, and retains only claims supported and not contradicted by visible behavior. Across diverse personalization tasks and user-model backends it outperforms existing attacks, is validated on real systems using information confirmed to be retained, and keeps recovering user information under response-level defenses.

Sources: arXiv

A single poisoned snippet can plant the attacker's chosen CWE precisely into generated code

Retrieval-augmented code generation pulls external code, documentation and patches into the generation context, and prior work showed only that this trust boundary raises the general vulnerability rate. CodePoisonRAG makes it targeted: the chain combines CWE-specific vulnerability injection, which embeds a chosen source-to-sink flow while keeping the artifact task-aligned, with semantic mislabeling, which adds a false safety claim without repairing the behavior. The attacker touches none of the victim's knowledge base, retriever, re-ranker, generator, prompt or defense, and injects at most one artifact per anticipated task. Eighty-five poisoned artifacts covering ten CWE classes across Java and C give an aggregate corpus-poisoning ratio of 0.7%; across three generators all 85 appear in the Top-3 for their query, with attack success rates of 0.80 to 0.93. Against CodeGuarder, which injects vulnerability-specific security knowledge into the context, success stays between 0.40 and 0.71.

Sources: arXiv

Security pull requests from coding agents: about 4% of their output, merged less often, and rejected for length rather than for security

The authors run a large-scale empirical analysis over the AIDev dataset — more than 33,000 curated agent-authored pull requests from popular GitHub repositories — and, after keyword filtering plus manual validation, identify 1,293 confirmed security-related agentic PRs, roughly 4% of agent activity. Those PRs are not mostly narrow vulnerability fixes: the most common actions are supportive hardening — tests, documentation, configuration and better error handling. Compared with non-security PRs they show lower merge rates and longer review latency, which the authors read as heightened human scrutiny, with variation across agents and language ecosystems. Rejection correlates more strongly with a PR's complexity and verbosity than with any explicit security topic.

Sources: arXiv

CAPTCHA defenses assume a failing bot stays failing; an adaptive solver recovers every round

Vision-language models solve visual CAPTCHAs without task-specific training but approach each challenge from scratch, while specialized detectors answer in milliseconds only for categories they were trained on — neither improves with exposure. The authors pair a fine-tuned YOLOv8 detector with an open-weight VLM behind a confidence-based router, running entirely from screenshots and OS input events with no browser automation or DOM access, and reach 85.4% overall and 84.2% macro accuracy across 16 classes, above either component alone. The mechanism that matters: every VLM answer doubles as a training label, so the detector absorbs categories it was never trained for, typically after one or two encounters and with no human annotation. An operator can perturb images against the publicly released detector and drive its accuracy to 0%, but the perturbations leave the VLM untouched and its labels repair the detector. Under a year-long simulated arms race with monthly re-crafted perturbations the solver recovers every round, and a cheap teacher of roughly 70% accuracy hardens it as effectively as a perfect oracle.

Sources: arXiv

Lab announcements

Anthropic: Claude produced the first end-to-end computer-checked proof of Fermat's Last Theorem in 11 days (Sep 4)

Claude worked largely autonomously in Lean, writing 13 million lines of Lean code — more than five times the size of Mathlib, the community library the theorem builds on — and proving 30,300 theorems along the way, 29,500 of which are used in the final proof. Dozens of Claude agents divided the work of defining concepts and proving intermediate theorems; human input was limited to occasional high-level priority hints from Anthropic researcher Tianyi Peng. Kevin Buzzard confirmed the proof carries no assumptions beyond the axioms of mathematics. The boundary is worth stating: what is new here is the verification, not the mathematics — the proof follows Darmon, Diamond and Taylor's simplified version of Wiles, and Anthropic itself distinguishes it from work that produced novel mathematics. The post also notes that a number of initial attempts failed, with agents quickly losing track of project state and ceasing to collaborate.

Sources: Anthropic Research

[Added detail] OpenAI: GPT-6 Astra formally launched (Sep 3)

The launch post describes Astra as state of the art across computer use, coding, cybersecurity and science, and as the company's most intelligent and aligned model. That the model reached the "Critical" cybersecurity capability threshold under the Preparedness Framework was announced earlier.

Sources: OpenAI News