AI Frontier · Sep 7, 2026

Covering September 1-7. Two of today's papers independently say the same thing out loud: an agent's compromise rate is not a fixed property of the model, it is a function of the attacker's compute. Repeat-After-Me reaches 47% on commercial VLMs through black-box adaptive search, and the paper formalizing indirect prompt injection as test-time search measures success rising monotonically with the attacker's budget. On that finding, any agent security number that does not state the attacker's search budget is incomplete.

Research

Visual prompt injection can now force a parseable tool call, and it transfers

Image-domain prompt injection has been far weaker than the text domain against frontier commercial VLMs, because materially harmful output requires a long, format-compliant target string — a native tool call with exact function names and arguments. Repeat-After-Me closes that gap with black-box adaptive search: under a realistic setting where the benign user prompt is semantically unrelated to the injected task and does not authorize it, attack success exceeds 80% on open-weight models including Qwen3.6-27B and 47% on commercial frontier models including GPT-5.5. Transfer is the more awkward part: injections optimized against one surrogate retain 43-46% of their success rate on two commercial victims, and cross-sample transfer retains 64-66%. The authors ran the chain in a real OpenClaw deployment — in a default Discord configuration, an untrusted user need only post an image.

Sources: arXiv

Treat indirect prompt injection as a search problem and "attack success rate" stops being a property of the model

The authors formalize indirect prompt injection as a test-time search over an attack surface jointly induced by the environment, the user task and the injection task, then build an agentic attacker with a search harness: environment reconnaissance, structured reasoning over attack strategies, and adaptive evaluation using the victim agent's own feedback. More attacker test-time compute yields better vulnerability discovery and exploitation; ablations show explicit strategy management is what keeps the search from going redundant and losing its gains at larger budgets. Their conclusion is that agentic security evaluations must characterize the attacker's search procedure and compute budget rather than treating success as a budget-independent property of the victim.

Sources: arXiv

Deleting the memory record and making the agent forget are different operations

A long-running agent accretes more than a transcript: compressed summaries, plaintext memory, pending tool plans, and — under every serving API — a KV cache. Today's forget operations delete one plaintext memory record and stop, leaving every artifact derived from the revoked information intact. Modeling the runtime as a deterministic transition system, the authors prove the trajectory suffix after the target's injection step τ is irreducibly tainted without token-level attribution, that exact unlearning requires at least T−τ+1 recomputed transitions, and give Provenance-Guided Selective Replay, which attains that bound. The audit across three agent suites, nine baselines and three model families is unflattering: memory deletion leaves leakage unchanged, instruction-based forgetting collapses under elicitation probes (Leak@probes = 1.00), and even source redaction still acts on the revoked preference in 80% of episodes.

Sources: arXiv

A split-training setup passed its privacy evaluation, and leaked through the return channel nobody tested

Two-node split LLM training: the trusted local node sends protected activations to an untrusted cloud node, the cloud node returns its output, and the local node, holding the private loss, returns the output gradient. The frame the cloud node receives mixes real rows with decoys, and the loss ignores the decoys — so the decoys' gradients are exactly zero, and the pattern of zeros names which rows were real. The measurement protocol was fixed in advance: a leak injected at known strength to prove the instrument can see one, a shuffled-label control to prove it does not report absent leaks, and a threshold set before the runs. Across nine seeds the zeros identified the real rows on every frame, 4,096 of 4,096 per run. An attack on frame contents recovered about one extra token per hundred over a constant-guess baseline (+0.65 to +1.50 percentage points), while the shuffled controls recovered nothing. The sentence that matters comes after: every one of these runs passed the forward-channel privacy check and the quality check, and failed that same check once the returned gradient was included. Clipping and noising each gradient row closes it for about 0.01 nats of held-out cross-entropy.

Sources: arXiv

Individually correct security mechanisms do not compose, and the break is at the component boundary

Agent systems increasingly stack provenance tracking, authorization, policy enforcement, protocol adapters and execution controls, and the authors argue these individually correct mechanisms do not necessarily compose into an end-to-end secure system: security-critical context gets dropped, widened, rebound or reinterpreted as actions cross component boundaries. They name the failure mode security-context discontinuity and present CONTINUITY, which models each component with an assume-guarantee contract and carries authenticated security context across transitions through signed root grants, provenance commitments, role-bound transition receipts, bounded typed releases, transformation witnesses and effect-bound execution permits. The requirement is end-to-end consequence integrity: every realized external effect must be backed by a valid, current authorization witness linking principal, task, provenance, delegation, policy state, canonical action and finality boundary. Across 2,560 parameterized attack instances spanning 32 fault classes in four application domains, the full configuration commits no harmful external effect while completing all 700 benign tasks and escalating all 200 ambiguous cases.

Sources: arXiv

Spear-phishing personalization has a measurable payoff, and it cuts both ways

180 U.S. working adults produced 1,436 valid evaluations of AI-generated phishing emails built at four cumulative levels of information: workplace only; plus recipient name and job title; plus job responsibilities; plus coworker and shared-project context. In a sensitivity analysis, convincingness rose 2.40 points per level on a 0-100 scale, and the odds of expressing click intention rose 28% per level. Qualitative coding explains why detail is double-edged: details matching a participant's role and routine supported credibility, while incorrect, vague or channel-inappropriate details raised suspicion. A post-hoc descriptive analysis found messages from a named person referencing a supplied coworker scored higher, and drew more click intention, than messages sent in a department's or entity's name. One observation for defenders: among participants who did not express click intention, investigation stayed common while reporting declined and deletion rose — these messages are more likely to be quietly deleted than escalated.

Sources: arXiv

Lab announcements

OpenAI: An Alien Mind (September 6)

Jakub Pachocki reflects on increasingly capable AI and the challenge of keeping it aligned, calling for stronger safeguards and international coordination.

Sources: OpenAI

OpenAI: Research acceleration, the view from inside (September 6)

An account of how coding agents are reshaping AI research inside OpenAI, with early data on agent usage, experiment velocity and task complexity.

Sources: OpenAI

OpenAI: Supporting independent journalism in Ukraine (September 7)

OpenAI, AIRPPU and WAN-IFRA launch an AI program for Ukrainian news organizations.

Sources: OpenAI