CS Paper Radar

When Reviews Gate Access: Collusion, Sabotage and the Scale of Peer Review in the AI Submission Surge

Yuchen Li, Xintong Li, Ken-Tye Yong, Nguyen H. Tran

cs.MA ·submitted 2026-11-14 00:55 UTC · v1

Abstract

AI tools have helped push submissions to the largest machine-learning venues to tens of thousands per cycle. Venues have answered with rules that tie access to review: per-author caps, mandatory reciprocal reviewing, reviewer eligibility earned through accepted papers, and assignment based on reviewer bids. Under these rules a reviewer is also a competitor of the authors. We study two deviations that the rules make attractive. Sabotage lowers a competitor's score. Collusion raises an ally's score. We build an agent-based conference calibrated to public statistics of NeurIPS and ICLR and run 6,220 simulated conferences. Four results stand out. First, the two deviations do different harm and amplify each other. Collusion changes who is accepted, sabotage erodes what scores say about quality, and rings that may also sabotage gain more than rings that only collude. Second, under random assignment the chance that collusion changes a decision is set by one exposure index, $E = Rm/A$, the expected number of allies among a paper's reviewers. Growth therefore protects random assignment, but bidding removes the protection. In a pool of 3,000 reviewers a 20-member ring turns 0.8% of its papers into acceptances under random assignment and 71% when 30% of allies' bids are granted. Third, when agents choose their allies every cycle, caps at the inner layers of the social brain keep collusion nearly harmless. Without a cap agents ally with half of their area and accepted value falls by 35%. Fourth, reviewer eligibility tied to acceptances turns sabotage into a weapon of incumbents, and a longer eligibility window widens the gap between incumbents and newcomers only when sabotage is possible.

Closest papers

Nearest neighbours by meaning across the whole index.

CABAL: Multi-Agent Simulacra for Tracing the Effects of Collusive Bidding in Peer Review

cs.AI cs.CY ·2026-09-04 ·Jicheng Zhou, Kemou Li, Kahim Wong, Zheyuan Li, Zhuan Shi +3 more
hot direction similarity 0.74
Abstract

Recent reports during the AAAI-27 review cycle highlight the risk of reviewers coordinating bids for reciprocal assignment advantage. Prior work treats bidding, reviewer assignment, and review manipulation as separate stages, leaving the lifecycle effects of collusive bidding unclear. Real-world analysis is further constrained by typically unobservable collusive intent and the lack of counterfactuals for the same conference. Motivated by this gap, we introduce \alg, an end-to-end multi-agent simulacra framework for studying reviewer assignment integrity by holding the conference environment fixed and configuring LLM-driven reviewer agents with honest or collusive policies. We further develop an affinity-guided collusive bidding strategy that uses mutual reviewer-paper affinities to construct collusion rings and select target papers, producing expertise-consistent rather than arbitrarily targeted attacks. Controlled experiments show that collusive bidding more than doubles target-paper capture and that assigned colluders score target papers about two points higher than honest co-reviewers, while conference-wide effects remain comparatively modest. Evaluated bid-phase detectors provide only limited evidence of collusion: in a fixed-triplet detector stress test, native positive-bid graphs are confounded by benign affinity, while a Very-High-only diagnostic view enables precise but low-coverage local recovery.

0.49

Change the Funnel, Not Who May Submit: Engineering Peer Review for the Flood

cs.LG cs.CYcs.DL ·2026-10-31 ·Mikhail L. Arbuzov, Lee Mosbacker, Sisong Bei
similarity 0.68
Abstract · 34 pages, 7 figures, 18 tables

Every reviewer knows the feeling: more papers each cycle, three full reviews for each of them, and a suspicion that near the accept line the outcome is luck. The numbers bear it out. NeurIPS received 30,709 submissions in 2026, more than three times as many as in 2021, and reviewing them took some 184,000 expert hours, or 92 years of researchers' time. Three-quarters of those hours went to papers that were rejected. And when NeurIPS twice had the same papers reviewed by two committees, the second rejected half of what the first accepted. Conferences are responding by limiting who may submit. We simulated that answer, and an alternative, in a model calibrated to the NeurIPS experiments. The limits save little and select no better, and those based on a past acceptance fall mostly on new authors. The alternative changes how papers are judged, spending full reads only where a decision turns on them. In a staged funnel, an automated quality check first turns back papers that are not ready. The rest are ranked by their value, from short summaries, by readers that each submission brings. Only the finalists get a full read, which checks their claims and settles the final ranking. The staged funnel needs a little over half of today's expert hours, accepts 95.5% of papers in the top 3-10% (against 74% today), and makes nearly every top-0.5% paper a spotlight or oral. And because every paper brings its own reader, the funnel keeps pace with the flood.

0.32

What Do Collusion Probes Read? Three Pitfalls in Evaluating Multi-Agent Collusion Monitors

cs.LG cs.AIcs.CLcs.MA ·2026-11-07 ·Mainul Hasan Mohsin, Mohammad Shahidur Rahman
benchmarkcode/data similarity 0.64
Abstract · 30 pages, 5 figures. Code: https://github.com/mainul3195/collusion-probe-pitfalls ; data: https://huggingface.co/datasets/mainul3195/collusi…

Activation probes look like near-perfect monitors for secret collusion among large language model (LLM) agents. We show that they mostly detect that an agent was told to collude, not that it decided to. Our benchmark (2,500 committee and 1,476 code-review episodes) adds agents who receive the collusion instruction and refuse it, separating instruction from decision. Published detectors barely tell colluders from these refusers, and our best probe separates them mostly by reading the instruction: without it, agent AUROC falls from 1.00 to 0.61. Two further pitfalls inflate reported results: scored after the vote, a rule that only reads the votes reaches AUROC 0.990 but finds nothing before it; and in 57% of attempts no colluder argues for the target in public. The decision is still there: a probe trained on agents' own choices reads it (AUROC 0.73 without the instruction), and probes find colluders whose goal was fine-tuned into the weights (AUROC 0.995; pre-registered). We turn the controls into a protocol for testing any collusion monitor, and adapt scan statistics into an online monitor for groups of any size.

0.27

Learning Checklists to Reduce Lottery Effects in Scientific Peer Review

cs.DL cs.AIstat.AP ·2026-10-14 ·Xuanyou Liu, Ziyang Guo, Jason Hartline, Jessica Hullman
similarity 0.61
Abstract

Reviewer lottery error describes error due to a small set of peer reviews attending to only some of the issues with a paper that a larger pool of eligible reviewers would raise. We show this error can be reduced by surfacing a checklist of criticisms learned from a historical review corpus. Our approach generates a tree of checklists representing coarsenings of the atomic review claim space, then selects the coarsening that maximizes a theoretically-motivated measure of correlated agreement among reviewers. We realize this approach with an LLM-assisted pipeline and test it on NeurIPS 2021 and ICLR 2022 review data. In simulated review with LLM reviewers, the learned checklist reduces the mean squared error (MSE) of a three reviewer committee by 58% on NeurIPS and 21% on ICLR, and on both venues the checklist with the highest correlated agreement has the lowest error for committees of three or more. In a human study with 20 participants, the learned NeurIPS checklist lowers the MSE of a paper ranking, measured against the NeurIPS 2021 order, from 0.48 to 0.22.

0.20

Who Pays for Open Review? Visible Author Reputation and Its Effect on Ratings

cs.DL cs.AI ·2026-09-07 ·Qinghua Zhao, Xinyu Chen, Yanhui Yang, Tengfeng Sun, Junfeng Liu +1 more
similarity 0.61
Abstract · 11 pages, 6 Tables

An OpenReview bug in November 2025 broke anonymity at several conferences and prompted calls for open review, which motivate us to ask what shifting from blind to open would mean for authors. Analyzing over 18,000 reviewed submissions to ICLR 2026, split into de facto open and blind groups by arXiv preprint timing, we find that ratings rise with author reputation under both mechanisms, with a steeper slope under open review that is statistically significant, and that the open-blind difference is concentrated at the borderline ratings. The pattern holds across five reputation proxies (including institution, h-index, and citation count), three author-aggregation rules, and five definitions of the open window. A controlled simulation with five AI models as reviewers, holding the manuscript fixed and varying the author reputation, reproduces the effect. With claude-opus-5 as the reviewer, for example, rating rises by 0.5 points as the author moves from low to high reputation.

0.30

The Helpful Agent Problem: When Emergent Cooperation Among LLM Agents Bypasses Intended Workflows

cs.AI cs.CLcs.MA ·2026-10-22 ·Moumita Choudhury, Amit Sarker, Shlomo Zilberstein
benchmark similarity 0.61
Abstract

LLM agents are increasingly deployed in collaborative settings where they pursue separate tasks while sharing resources and operating under review or oversight processes. Such settings create safety risks not only when agents are malicious, but also when they are simply trying to help. Prior work has largely studied adversarial or explicitly collusive behavior; we show that benign agents pursuing separate objectives can spontaneously cooperate in ways that bypass intended workflows. We call this the \emph{helpful agent problem}. We study long-horizon shared-resource environments in which agents sometimes depend on resources owned by their peers and cross-owner changes are normally mediated by a reviewer. When review becomes unavailable, agents begin making changes on one another's behalf despite having no shared reward or explicit incentive to cooperate. Crucially, these actions remain within each agent's local permissions while circumventing the system's intended oversight structure. We introduce \textsc{FavorBench}, a controlled benchmark with measurable downstream side effects, and adapt \textsc{CooperBench} to test the phenomenon in realistic repositories. Across two benchmarks and four models, we find that out-of-workflow favors emerge frequently. Further, in \textsc{FavorBench}, favors from at least three models cause significant harm to parties outside the interaction. These results reveal a distinct multi-agent safety challenge: cooperation itself can create pathways around system-level safeguards.

0.15

Winning Reviews, Losing Discovery? The Co-Evolutionary Game of AI Research and Peer Review

cs.GT econ.GN ·2026-11-07 ·Hongze Liu, Mengru Wang, Zhiyuan Zhang, Yao Liu
similarity 0.60
Abstract

Autonomous research systems can produce convincing manuscripts whose scientific value is difficult to verify, while automated reviewers can reward the appearance of completeness. We examine how this gap can become a self-reinforcing threat to the research community. In a two-population evolutionary game, authors choose between acceptance-oriented and exploratory research, and reviewers choose between proxy evaluation and careful assessment. Cheaper acceptance-oriented production can increase review pressure, making proxy assessment more attractive; proxy assessment can in turn strengthen the incentive to produce reviewer-compatible work. We derive conditions under which this feedback sustains a stable regime that rewards publication success while reducing the scientific value of accepted work. An analytical basin boundary explains why the same institutional rules can support either this regime or exploratory research with careful review. Numerical illustrations show how publication counts can conceal scientific loss, how faster adjustment can reinforce existing practices, and why partial reform may fail once the cycle is established. Careful audits and reviewer rewards can reverse the feedback, while temporary support can produce lasting recovery if it moves the community across the basin boundary. The model identifies a warning mechanism rather than forecasting a particular conference: a review system can remain operational and privately rewarding while becoming less effective at supporting discovery. Its institutional implication is to reward independently validated contributions and consequential checking, rather than accelerate paper production and scoring alone.

0.06

Emergent Collusion in Long-Horizon LLM Agent Interaction

cs.AI cs.CL ·2026-09-21 ·Xinrui Shi, Yanzhe Zhang, Diyi Yang
highly cited similarity 0.60
Abstract · 53 pages, 13 figures

LLM agents are increasingly deployed in collaborative settings, yet long-term interaction may give rise to undesirable coordination. We study the emergence of collusion in a long-horizon multi-agent environment: two agents repeatedly complete individual tasks, share task logs, verify each other's work, and receive rewards. We introduce realistic constraints under which higher reward is attainable only by violating the verification protocol, and find that agents increasingly deviate from the protocol over repeated interactions. Collusion emerges in 94% of trajectories across 10 models, and more capable models within the same family reach it earlier. Controlled peer interventions show that collusion is shaped by peer behavior, while ablations reveal additional effects of the verification feedback agents receive, their interaction history, and the reward structure. In particular, restricting the amount and scope of interaction history available to agents reduces collusion. Overall, our findings show that long-horizon interaction can reshape how agents coordinate in ways that create safety risks.

0.61