CS Paper Radar

What Do Collusion Probes Read? Three Pitfalls in Evaluating Multi-Agent Collusion Monitors

Mainul Hasan Mohsin, Mohammad Shahidur Rahman

cs.LG cs.AIcs.CLcs.MA ·submitted 2026-11-07 09:53 UTC · v1
benchmarkcode/data

Abstract

Activation probes look like near-perfect monitors for secret collusion among large language model (LLM) agents. We show that they mostly detect that an agent was told to collude, not that it decided to. Our benchmark (2,500 committee and 1,476 code-review episodes) adds agents who receive the collusion instruction and refuse it, separating instruction from decision. Published detectors barely tell colluders from these refusers, and our best probe separates them mostly by reading the instruction: without it, agent AUROC falls from 1.00 to 0.61. Two further pitfalls inflate reported results: scored after the vote, a rule that only reads the votes reaches AUROC 0.990 but finds nothing before it; and in 57% of attempts no colluder argues for the target in public. The decision is still there: a probe trained on agents' own choices reads it (AUROC 0.73 without the instruction), and probes find colluders whose goal was fine-tuned into the weights (AUROC 0.995; pre-registered). We turn the controls into a protocol for testing any collusion monitor, and adapt scan statistics into an online monitor for groups of any size.

Comment
30 pages, 5 figures. Code: https://github.com/mainul3195/collusion-probe-pitfalls ; data: https://huggingface.co/datasets/mainul3195/collusion-probe-pitfalls ; archive: doi:10.5281/zenodo.23636508

Closest papers

Nearest neighbours by meaning across the whole index.

Pre-Crime for Agents, at the Last Second: Probing When a Recruited Agent Gives In

cs.AI ·2026-10-29 ·Leonidas Raghav
code/data similarity 0.69
Abstract · 49 pages, 11 figures. Code: https://github.com/leonidas1712/precrime-agents

When AI agents share channels and credentials, one agent can press another to break its operator's rule, a first step towards collusion. Output filters check only what an agent has already written. Can an overseer tell beforehand that an agent is about to give in? We show that a linear probe on the agent's activations can, in a controlled multi-agent setting. Across six scenarios, a peer presses a target for up to 20 rounds to reveal a secret, such as a deploy token; the target often refuses, then gives in. Read at each model call before its output is written, the probe picks out the one that leaks. On held-out Qwen3-32B episodes, at a threshold set to flag 5% of benign calls, it catches 82% of leaking calls. At a similar flag rate, a zero-shot decision-model monitor catches 26%. On gpt-oss-20b, a probe on its reasoning reaches AUROC 0.91. A bag-of-words classifier trained on the same calls comes close, so most of the gain comes from training on labels, not activations. The probe also flags permitted disclosures: it reads the coming disclosure more than the violation. The warning is short-range, and halting on any alarm would stop most benign episodes, which a second text check does not fix. A warning raised before generation could let an overseer stop an agent that is about to give in, ahead of others acting on its messages or recruitment spreading through a swarm.

0.35

What Standard Evaluations Hide About Detectors of Hijacked LLM Agents: Rare Attacks, Unseen Tools and Benchmark Transfer

cs.CR cs.LG ·2026-10-30 ·Mahmoud Abbasi, Juan M. Núñez V., Amin Shahraki
benchmarkcode/data similarity 0.65
Abstract · 14 pages, 4 figures, 6 tables. Code, data and every monitor call: https://github.com/Mahmoud-Abbasi-svg/agent-trace-detector-eval

Detectors can flag hijacked large language model (LLM) agents from the traces of the tools they call. In the eleven evaluations we audited, attacks or anomalies make up 16% to 86% of the test samples, no evaluation varies that share, and six report no test on data from a second benchmark or source. We ask what such evaluations hide. On the AgentDrift and AgentDojo benchmarks we test trained detectors that read tool identities, behavior, call sequences, step embeddings or text, and a released LLM action monitor, AlignmentCheck, with thresholds fixed on training data or set by the monitor's released decision rule. First, at a 2% attack rate a detector with an area under the receiver operating characteristic curve (AUROC) of 0.879 reaches a precision of 0.076, and the mix of negatives alone moves that precision between 0.039 and 0.117. Second, as tool coverage, the share of test tool calls whose tool was seen in training, falls, detector rankings cross near 50% coverage and reverse below it, because unseen tools map to inputs that training never reached. Dropout on tool inputs largely removes the reversal. Third, no detector trained on AgentDrift ranks AgentDojo traces better than chance (AUROC 0.47 to 0.52), including one that reaches 0.996 within AgentDojo. The monitor, trained on neither, reaches 0.930, but with its open backbone its released decision rule would still mostly raise false alerts at a 2% attack rate (precision 0.096). We turn these findings into a checklist for evaluating trace detectors.

0.28

Model Organisms of Collusion: Corrective Transfer in Untrusted Monitors

cs.AI ·2026-10-22 ·Juan-Pablo Rivera, Ari Spiesberger, Juan J Vazquez, Arun Jose, Axel Ahlqvist +4 more
similarity 0.64
Abstract · 33 pages, 6 figures, 15 tables

AI control protocols aim to get useful work from models that may be misaligned by monitoring their behaviour. One approach uses a second copy of the same model as the monitor. This avoids relying on a weaker reviewer the model could outsmart, but risks collusion: the model, now monitoring itself, may cover up its own misbehaviour. One remedy is to fine-tune the monitor on whatever examples of misbehaviour the control team has caught. We ask whether this training also removes collusion that the control team has not yet found. We fine-tune Llama-3.3-70B and Qwen2.5-72B to install five policy-monitor behaviour pairs, correct the monitor on 300 honest reviews of one caught collusion, and measure collusion on the other four in unseen prompt formats. Averaged over the three seeds of each family, correction reduces held-out collusion from 91% to 14-20% under three of the four methods we compare (individual seeds 10-32%), while a control fine-tuned on the same number of unrelated examples leaves 56-93%. Family-average accuracy on control items changes by at most five points and capability is preserved, apart from a drop in mathematical accuracy on one Llama seed. Under the tested training configuration, methods with similar caught-task accuracy differ in held-out transfer, from 58% to 83% average reduction, so the caught behaviour alone does not reveal how far a correction reaches. Training on a known failure removes it almost entirely (1-2% remaining), yet 14-20% of unknown failures persist, so near-perfect correction of the caught behaviour overstates how much collusion has been removed.

0.32

When Reviews Gate Access: Collusion, Sabotage and the Scale of Peer Review in the AI Submission Surge

cs.MA ·2026-11-14 ·Yuchen Li, Xintong Li, Ken-Tye Yong, Nguyen H. Tran
similarity 0.64
Abstract

AI tools have helped push submissions to the largest machine-learning venues to tens of thousands per cycle. Venues have answered with rules that tie access to review: per-author caps, mandatory reciprocal reviewing, reviewer eligibility earned through accepted papers, and assignment based on reviewer bids. Under these rules a reviewer is also a competitor of the authors. We study two deviations that the rules make attractive. Sabotage lowers a competitor's score. Collusion raises an ally's score. We build an agent-based conference calibrated to public statistics of NeurIPS and ICLR and run 6,220 simulated conferences. Four results stand out. First, the two deviations do different harm and amplify each other. Collusion changes who is accepted, sabotage erodes what scores say about quality, and rings that may also sabotage gain more than rings that only collude. Second, under random assignment the chance that collusion changes a decision is set by one exposure index, $E = Rm/A$, the expected number of allies among a paper's reviewers. Growth therefore protects random assignment, but bidding removes the protection. In a pool of 3,000 reviewers a 20-member ring turns 0.8% of its papers into acceptances under random assignment and 71% when 30% of allies' bids are granted. Third, when agents choose their allies every cycle, caps at the inner layers of the social brain keep collusion nearly harmless. Without a cap agents ally with half of their area and accepted value falls by 35%. Fourth, reviewer eligibility tied to acceptances turns sabotage into a weapon of incumbents, and a longer eligibility window widens the gap between incumbents and newcomers only when sabotage is possible.

0.22

Telling FIBS: Probes Effectively Detect Sabotage and Catch Unverbalized Deception

cs.LG cs.AI ·2026-10-08 ·Oskar J. Hollinsworth, Alex F. Spies, Tigist Diriba, Adam Gleave, Chris Cundy
datasetcode/data similarity 0.62
Abstract · 10 pages main text, 119 pages total; 26 figures, 28 tables. Code and data: https://github.com/AlignmentResearch/fibs-probes

Recent incidents have highlighted the challenge of monitoring LLM agents and the danger of models deceiving people. We show that white-box deception detection via probes can be scaled up to frontier monitoring settings by collecting the largest deception dataset to date for training probes and introducing a novel probe architecture which can aggregate information across many layers and tokens. Our probes surpass black-box monitors on agentic coding benchmarks, achieving 98.3% AUROC in SHADE-Arena and 98.0% in BashArena (vs. 97.7%, 97.6% resp. for action-only text monitors), and show improved efficacy as the underlying model is scaled up. We further test probes on several cases where deception cannot be determined from the context alone; in these cases of introspective deception, the ground truth can only be determined through careful elicitation or knowledge of a model's training data. In one such evaluation, we show that probes can distinguish transcripts containing a model's true hidden goal from other goals with an AUROC of up to 99.7%. Our probes also detect deception on prominent open-weight models which lie about politically sensitive topics when put under pressure. We release our training dataset, dubbed FIBS, to help drive frontier deployment of effective probes, and encourage the community to expand upon it with further examples of deception and sabotage.

0.51

You Can't Escape Your Activations : Evaluation Awareness and Multi-Agent Monitoring

cs.MA cs.LG ·2026-09-02 ·Jaee Ponde, Aritra Das, Mihir More, Debayan Gupta
similarity 0.61
Abstract · 18 pages, 11 figures, 2 tables

LLM agents are increasingly deployed in multi-agent systems, where they can collude while keeping their actions benign. Output monitors designed to detect such collusions can be fooled by obfuscation and steganography, motivating the use of probes trained on internal activations. However, these probes are usually evaluated on agents that do not know they are being watched. We study how activation-based detection changes when agents are explicitly informed that their internal activations are being monitored, and when they additionally receive feedback from the monitor. We keep the models, probes, and thresholds fixed and change only what the agents are told: nothing (baseline), that an activation monitor is present (aware), or that a monitor is present together with the previous round's score (feedback). We test two games, a four-agent blackjack game and a two-agent Simmons prisoners game, using Qwen3-32B-AWQ and GPT-OSS-20B in homogeneous and mixed pairings. Telling agents about the monitor does not hide them. The best probes stay accurate in all three conditions, and the agents keep colluding.

0.22

Multi-Agent Collusion and the J-Space

cs.CL ·2026-11-01 ·Marco Molinari, Gabriele Bettineschi
benchmarkhot direction similarity 0.61
Abstract

LLM agents are increasingly deployed in Multi-Agent Systems (MAS), and they introduce the risk of coordination among agents that might escape human oversight. The J-space has been used to uncover deception in single LLMs, but its use for uncovering and steering collusion between agents remains unexplored. We show, with interpretability experiments on DeepSeek-V4-Flash (284B) on the NARCBench collusion benchmark, that colluding agents' J-space holds the collusion itself: before replying, 86% of colluders represent misrepresentation, concealment or remorse. In a steganographic card-counting task, the receiver's J-space holds a hidden count, and causal interventions alter its count. We also introduce the Jacobian Pair Contrast (J-PACT), an unsupervised method for detecting collusion between agents. We evaluate J-PACT on four open-weight models: GPT-OSS-20B, Llama-3.3-70B-IT, Qwen3-32B, and DeepSeek-V4-Flash (284B). J-PACT retains 85% of the above-chance AUROC of supervised NARCBench probes, matches them on steganographic scenarios, and names the colluding pair up to its complement in 99% of detected runs.

0.28

Explore or Confirm? LLM Agents Misallocate Measurements Under Noise

cs.AI cs.MA ·2026-10-18 ·Abdul Karim, Matt Keon
benchmarkcode/data similarity 0.61
Abstract · 38 pages, 12 figures. Code, data and environment: https://github.com/Xstendyl/wellocate

As model reasoning gets cheaper, the physical experiment becomes the slow and costly step in discovery. Each experiment returns a noisy reading: a high value can mean a good candidate or a lucky draw. With a fixed budget, an AI agent must keep choosing between exploring a new candidate and confirming a promising one by re-testing it. We measure that choice in Wellocate, a simulated assay campaign with known true values. It is built on a generated landscape and on two real protein datasets, GB1 and PhoQ, scrambled so no model can answer from memory. We test five language models as minimal agents, with no tools and no memory beyond the history in each prompt. They misallocate in both directions. Three over-confirm, spending most of the budget re-testing. On the generated landscape one under-confirms, trusting single readings, and there every model ends with a worse final pick than a basic scripted strategy. Each failure has its own cause. The over-confirmers run short of new candidates worth trying: a list of untried candidates cuts their re-testing and, on real protein data, makes two of them pick much better. A running summary of their readings or a description of the landscape changes it by at most five percentage points. For the under-confirmer, re-testing its top candidates in a final round, replayed on its own campaigns, recovers only about a quarter of what it loses by picking the wrong candidate. This suggests confirmation has to run through the whole campaign. We release Wellocate as a benchmark and reinforcement-learning environment with frozen splits and scripted baselines. With it, anyone can measure how an agent splits its budget before trusting it with real experiments, and train agents to split it better. Code, data and the environment are available at https://github.com/Xstendyl/wellocate.

0.41