AI Frontier · Sep 14, 2026

Covering Sep 8 to 14. Two unrelated papers this week point at the same place: the check is asking the wrong question. Scanners at agent skill registries decide whether an artifact is malicious, while the chemistry and materials agent benchmark shows hazardous output arriving only after tool calls and memory have been walked off course step by step. What both need to ask is whether this action is permitted here, by this operator, right now.

Research

Scanners overlap on at most 10.4% of their positives, and 705 skills they all rate clean still instruct a prohibited action

Three measurements over 66,192 public ClawHub skill versions. First, 705 skills from 135 distinct publishers are rated clean by every scanner and by the registry's own judge, yet instruct an action prohibited by CIS Control 2.7 and NIST SP 800-53 CM-11; a hand audit of 100 puts the authors' detector at 92% precision and found no marker of malicious intent — these are violations, not malice. Second, of 144 commands a live agent executed in a sandbox while following real skill documentation, 34.7% carried a consequence class absent from that document. Third, across 53 cleared skills that document an action no clean record earns, the agent reached for one in 23 of them, and the gate stopped all 23.

Two limits worth reading for yourself: 506 of the 705 come from a single publisher, so this is not 705 independent problems — the authors report the distribution alongside the count — and the 92% precision is their own detector plus a hand audit, not third-party review. The scanner figures they cite come from OpenClaw's security team: at most 10.4% overlap on the combined positive set, with 81.9% of flagged skills caught by one scanner alone.

Sources: arXiv

Chemistry and materials agents released complete hazardous procedures in 25.6% of runs, and 9.2%–22.5% after defenses

ChemMat-AgentSafetyBench is built from 432 fixed harmful case specifications spanning eight hazard classes, three scenario shells and four tool-and-memory environments, plus five online long-horizon attacks: intent hijacking, tool chaining, objective drifting, task injection and memory poisoning. The concrete language of each online attack is generated at runtime from the evolving trajectory, so it is not counted in the static benchmark size. In the four-model main experiment with a fixed attacker, agents released complete hazardous synthesis or preparation procedures in 25.6% of runs; replacing the attacker model gives mean success rates from 18.4% to 26.5%, so the risk is not an artifact of one attacker. Input- and state-level defenses adapted from general-purpose agent safety, together with candidate checks designed for chemistry and materials, bring complete-path release down to between 9.2% and 22.5% — reduced, but not to a level anyone can ignore.

The authors' framing is that the safety question has moved: not whether a model answers a hazardous question, but whether an agent releases a hazardous protocol through a tool-mediated workflow.

Sources: arXiv

Asking several questions in one batch is itself a jailbreak

A harmful question that is reliably refused in isolation can elicit a harmful response when embedded in a batch of benign questions. The authors identify this as a distinct safety failure mode and argue it is not reducible to known in-context learning or long-context effects, explaining it from two angles: alignment signal weakening and refusal signal dilution. Across widely used open-source and frontier commercial models, batch prompting consistently achieves high attack success rates as a simple black-box attack. The mitigation they report as effective is batch-aware preference optimization.

What gives this weight in practice is that batch inference is the standard way to cut cost: a system that packs user requests together to save money is running this attack on its default path. Code is published.

Sources: arXiv

In agent pipelines, what governs privacy leakage is when values are restored

BodhiPromptShield targets prompt privacy risk that propagates beyond a single model call: raw user content enters retrieval queries, memory writes, tool arguments, OCR-derived text and logs, and every downstream copy inherits what the first write contained. It detects sensitive spans before they propagate, replaces each with a typed placeholder, a semantic abstraction or a secure symbolic token under a configured policy, and defers restoration to authorized execution boundaries.

Three results. Identifier propagation is controllable: residual exposure falls to 7.4% on AI4Privacy and 1.8% on PrivacyLens, and exact identifiers in the agent's final action fall from 13.7% to 2.1–3.1%. Restoration timing governs what every stage upstream of the authorized boundary sees: deferring it leaves 1.6% of protected values readable in the released context, against 51.0% when it is not deferred, for 0.11 helpfulness points. The third is the one the authors did not settle — measuring factual disclosure is harder: a word-overlap metric and an LLM judge both report that mediation leaves facts intact, and the two do not agree with each other, so the question stays open under their protocol.

Sources: arXiv

Auditable evidence for agent runs: tamper-evidence delivered, replay half delivered

Regulation (the EU AI Act, ISO 42001, NIST AI RMF) presumes records an independent party can check, while the traces agent-observability platforms produce are mutable: alterable undetected, with no recipe for re-executing them, and silent on whether captured secrets were removed. NovaFabric records a run into a portable Run Capsule (a fifteen-entity schema), sealed with a holistic DSSE signature, an RFC 3161 timestamp, a Merkle log and a redaction attestation. The authors say plainly that the contribution is integration rather than new cryptography — OpenTelemetry, DSSE/in-toto and W3C PROV.

The part worth copying down is the limits they report themselves. Mocked replay does serve every model response from the capsule (10/10), but it is offline with respect to models, not the network; only 2 of 10 tool-using workloads completed, and the gap is missing tool-response substitution. Tampering is rejected across all three tested classes. Declared-stream completeness is 0.652 (95% CI ±0.064, ten scenarios). A repaired rule pack redacts all 14 credential types and preserves all 9 decoys, and diff localises all 140 mutations. Blast-radius queries run at 45.5ms p99 over 10M edges. Third-party verification is, in their words, specified but not evaluated.

Sources: arXiv

Attribute inference with provenance: 86.7% success, with evidence cited for over 98% of predictions

Sensitive attributes such as age, income and occupation can be inferred by aggregating indirect cues across many ordinary posts, and LLM-based profilers automate that aggregation. What separates GraphProfiler is auditability: it represents each user's post history as a source-linked personal knowledge graph whose nodes and edges trace back to the originating post, and resolves attribute predictions to cited graph records and source texts. It reaches an 86.7% attack success rate on the eight-attribute SynthPAI benchmark, within two points of strong text-only baselines, and 84.6% on PANDORA, citing supporting evidence for over 98% of predictions.

A controlled ablation earns those citations their standing: removing the cited posts reduces attack success substantially more than removing an equal number of random posts. That is the point the authors are after — targeted privacy mitigation that rewrites or redacts only the few posts actually leaking an attribute, rather than perturbing an entire history.

Sources: arXiv