CS Paper Radar

Incremental Speciation in Evolutionary Red Teaming of Large Language Models

Onkar Shelar, Travis Desell

cs.NE ·submitted 2026-11-18 06:34 UTC · v1

Abstract

Speciation maintains distinct subpopulations in evolutionary search through similarity-based grouping and species-aware parent selection. For text-based individuals, niches depend on both the compatibility representation and the assignment operator that turns pairwise relations into species. We examine this design space in black-box evolutionary red teaming of large language models, comparing embedding and natural language inference (NLI) dissimilarities under representative-based and density-based speciation. Offline calibration on a 3,139-prompt archive shows that MiniLM embedding dissimilarity ($d_E$) and DeBERTa-v3-small NLI non-entailment ($d_{\neg E}$) produce different species structures at the same nominal threshold. Even with pairwise edge density matched to MiniLM, NLI density connectivity forms a few concentrated, mixed-topic species, with substantially more off-topic membership. These structural differences persist online under a matched evaluation budget. The endpoint top-50 median fitness favors incremental DBSCAN over leader-follower, but bootstrap intervals from 10 runs include zero, providing no reliable evidence of an advantage. Semantic diversity among top prompts declines in every configuration, with no consistent method advantage. The clearest difference concerns high-fitness population occupancy: MiniLM incremental DBSCAN maintains mature niches across seed topics, whereas NLI incremental DBSCAN attains the same highest observed median fitness while concentrating high-fitness prompts in variants of a few seed topics. Thus, representation-dependent connectivity predicts population structure more consistently than endpoint fitness. Species count alone does not establish quality-diversity, and more species do not necessarily preserve semantically diverse high-fitness prompts.

Closest papers

Nearest neighbours by meaning across the whole index.

When the Feature Pool Goes Algorithmic: Extending Mufwene's Ecology of Language Evolution to LLM-Mediated Exposure

cs.CL ·2026-08-21 ·Kunmei Han
similarity 0.60
Abstract

Mufwene's ecological model locates language evolution in competition among variants contributed by individual idiolects and in speakers' selection from linguistic material made available through interaction. Large language models (LLMs) complicate this architecture without requiring the locus of selection to move away from human speakers. This article argues that LLMs are best treated as distributional mediators: they aggregate language produced across human populations, transform its distribution through training and post-training, and redistribute model-specific outputs at scale. I call the resulting ecological process algorithmic reweighting of the speaker-accessible distribution: model mediation can alter the relative frequencies with which competing variants reach human selectors. Emerging evidence on model-specific linguistic profiles and lexical uptake is consistent with parts of this pathway, but does not establish inevitable convergence. Human social evaluation remains decisive: model-associated forms may diffuse and become conventionalized, become socially recognizable as 'AI-like' and subsequently avoided, or fail to diffuse in the first place. The proposal extends Mufwene's feature-pool ecology one step upstream of speaker selection and yields testable predictions about uptake, model-version effects, convergence, and social reversal.

0.38

The Multilingual Artificial Hivemind: Do All LLMs Think Alike in Every Language?

cs.CL ·2026-10-12 ·Xaver Raphael Davey, Shubhra Mishra, Anton Adelöw, Siyao Liu
benchmark similarity 0.60
Abstract

Text generated by Large Language Models (LLMs) often remains limited in diversity compared to that produced by humans. Recent advances demonstrate the presence of inter- and intra-model collapse in open-ended generation of English text, raising concerns about the long-term homogenization of LLM, and more importantly, human thought. However, our understanding of whether this homogenization extends to languages beyond English is limited. In this work, we investigate the role of language on output diversity. Specifically, we aim to understand whether multilingual LLM-generated text is also homogeneous, and whether this homogeneity matches what we observe in humans. Across 23 open-weight models and 13 languages, we find that the artificial hivemind effect is pervasive but uneven in its extent. The intra-model effect holds in three different embedding spaces and under a lexical metric that uses no embedding model, so it is not an artifact of how diversity is measured. The size of the homogenization gap is associated with a model's competence in a language, measured on a held-out translation benchmark, robustly across embedding spaces. Thus, we demonstrate a performance-diversity tradeoff in multilingual LLMs.

0.26

The Evolution Strategies Paradox: Better Models from Worse Perturbations

cs.AI ·2026-11-03 ·Roberto Dailey, Kajetan Schweighofer, Jamieson Warner, Conor F. Hayes, Elliot Meyerson +3 more
similarity 0.57
Abstract · 20 pages, 9 figures

How is it possible for perturbation-based methods like Evolution Strategies (ES) to optimize billion-parameter large language models (LLMs)? Classical search methods are highly dependent on finding good candidates within generated populations. However, when using ES to fine-tune LLMs, a phenomenon can occur where all offspring models consistently perform worse than the parent model, yet during training, the parent model consistently improves. This paradox occurs across models, tasks, and even modalities. This paper characterizes the paradox, then examines how the local reward landscape and properties of the ES estimator contribute to it: in high-dimensional spaces, exploration leads to worse performance, yet still highlights good directions; aggregating those directions then allows for making local improvements. The result can be seen as a new optimization paradigm where a principled interaction of noise and curvature produces the paradox as a byproduct of effective optimization. These findings provide guidance for taking advantage of perturbation-based methods in LLM fine-tuning and in high-dimensional optimization in general.

0.12

SEDIMA: Cross-Run Hierarchical Insight Memory for Evolutionary Search Agents

cs.NE cs.CL ·2026-10-01 ·Amirhossein Abaskohi, Mahdi Mostajabdaveh, Zirui Zhou
similarity 0.56
Abstract

Large language model (LLM)-driven evolutionary search is a powerful paradigm for automated program and algorithm discovery, yet existing systems are largely memoryless: each run explores from scratch, so agents repeatedly rediscover the same improvements and re-encounter the same dead ends. We introduce SEDIMA, a persistent hierarchical insight memory for evolutionary search agents. SEDIMA distills raw traces into natural-language insights, clusters them by semantic similarity using attention-weighted centroids, and retrieves relevant guidance to condition future mutations, accumulating transferable knowledge across runs and problems rather than within a single trajectory. As a drop-in module that leaves the search operators unmodified, SEDIMA improves average final performance by 5.5% on AlgoTune and 6.6% on ALE-Bench LITE under a fixed budget of 100 evaluated candidates. Under OpenEvolve, SEDIMA requires 32.3% fewer iterations on average to reach baseline-best performance across the five evaluated backbones.

0.26

Structure, Association, and Decision Value: Representation-Based Difficulty Estimation for Adaptive Inference in African-Language NLI

cs.CL ·2026-08-19 ·Toheeb Ogunade
similarity 0.56
Abstract · 21 pages, 3 figures, 10 tables. Submitted to MIRG-ICAIR 2026

We ask whether internal representation statistics can provide useful example-level difficulty signals for adaptive inference in multilingual African NLP, and find that they cannot in this setting. Studying natural language inference across 15 African languages with frozen off-the-shelf checkpoints, we report four results. First, AfriXNLI's English configuration shares 1,047 of its 1,050 examples verbatim with XNLI evaluation data, and one widely used NLI checkpoint scores 1.000 on that test split, consistent with XNLI test exposure. Because AfriXNLI is derived from XNLI, its English, French and Swahili configurations cannot serve as clean evaluations for XNLI-trained models. Second, parameter count does not reliably order capability across African languages: our larger checkpoint is better in seven languages and worse in eight, with no significant aggregate difference. Third, across three multilingual representation spaces, angular dispersion is consistently more language-determined than effective rank, so pooled correlations can inflate one and mask the other. Fourth, the association that survives language control depends on the target: effective rank predicts probability gain from escalation but not whether escalation changes the prediction, while cheap-model confidence shows the opposite pattern; the two targets correlate at only 0.655. Under the tested models, signals, and compute budgets, no evaluated signal makes adaptive routing preferable to always-expensive inference, although an oracle exceeds it by 11 accuracy points at 60% of the compute. Our central methodological finding is that a representation statistic can be statistically significant for one notion of computational benefit while being irrelevant to another, and therefore be a poor decision variable.

0.22

Scale, Drivers, and Limits of Representational Alignment Across Large Language Models

cs.CL cs.AI ·2026-10-10 ·Jialin Wu, Meenakshi Khosla
similarity 0.56
Abstract

Comparing large language models can reveal shared representational structure and systematic differences between them. However, aggregate similarity measures can obscure how agreement varies across geometric scales and which linguistic properties support it. We introduce two complementary methods to resolve where models agree and what supports that agreement. Standard representational similarity analysis (RSA) summarizes similarity across all stimulus pairs, potentially obscuring differences in alignment between nearby and distant inputs. Neighborhood-Weighted RSA (NW-RSA) addresses this limitation by weighting stimulus pairs according to their proximity in representational space, placing alignment from local neighborhoods to global geometry on a common scale. Counterfactual RSA (CF-RSA) tests which linguistic properties support alignment by comparing intact representations in one model with representations under controlled linguistic perturbations in another. Across diverse language models, we find strong agreement in coarse representational geometry but substantial divergence in local neighborhoods. Alignment is observed both across broadly sampled natural text and within linguistic equivalence classes that preserve particular aspects of meaning or structure. Counterfactual analyses reveal selective sensitivity: changes to logical scope reduce alignment substantially more than the tested changes to hierarchical syntax, event roles, or discourse structure. Alignment also decreases when sentence structure or lexical meaning is disrupted in word lists and jabberwocky sentences, suggesting that the regularities of natural language support stronger convergence. These findings refine claims of representational universality by showing that strong overall similarity need not imply agreement in how models represent finer linguistic distinctions.

0.15

Retrieval Disagreement as a Signal: Diagnosing and Steering Multilingual Dense Embedders

cs.CL cs.IR ·2026-10-13 ·Faeze Ghorbanpour, Shaghayegh Kolli, Jana Diesner, Alexander Fraser
similarity 0.55
Abstract

Multilingual dense retrievers place all languages in one shared space so that a question finds relevant documents, regardless of which language they are written in. We show that the same question, asked in different languages, retrieves substantially different documents, and this disagreement is not because of weak language understanding. Each language performs well on its own and ranks candidates consistently with the other languages; divergence appears only when languages share an index. We trace the mechanism: each language keeps its own ranking, but in the shared index, its correct answers are pushed far down, buried under non-relevant documents that are either in the query's own language or in English, depending on the model. We introduce disagreement-guided steering: for each language, we contrast the documents that other languages agree on for a question against those that language retrieves idiosyncratically, and take the difference of their representations as a language-specific direction, added to that language's queries at inference. Against baselines of language-mean removal, subspace removal, and per-language rotation, it most strongly reduces same-language crowding while remaining competitive on recall, trading off against mean removal, which better preserves ranking quality.

0.19

Taxonomy-aware distances between scholarly topic profiles via an exact simplex embedding

cs.DL ·2026-08-23 ·Dmitry Gubanov, Alexander Chkhartishvili
similarity 0.55
Abstract

Topic profiles represent publications, authors, and other scholarly entities as probability distributions over a fixed set of topics, but flat total variation treats every pair of distinct pure-topic profiles as maximally separated and therefore ignores taxonomic proximity. From a rooted weighted taxonomy, we derive a cardinality-normalized linear operator that maps the leaf topics to points in the original probability simplex and exactly realizes a normalized lowest-common-ancestor ultrametric under total variation. The operator is doubly stochastic and positive definite; within the class of nonnegative edge-cluster Gram operators, its normalization is uniquely determined on the reduced branching tree. Applying the same invertible operator to arbitrary topic mixtures yields a nondegenerate hierarchy-aware metric that contracts flat total variation, differs from the tree-Wasserstein distance on mixtures, and can be evaluated in O(|V|+L) time and memory without forming the dense matrix. In a frozen OpenAlex taxonomy with 4,516 terminal Topics, raw dissimilarities between Topic texts showed consistent ordinal alignment with taxonomic proximity, while only 3 of 253 calibrated internal nodes required monotonic correction. Encoder choice nevertheless affected individual height estimates. The framework exactly realizes a supplied weighted hierarchy; text is used only to initialize its node heights, and distances between scholarly topic profiles are then computed in the induced geometry.

0.24