CS Paper Radar

From Admission to Retention: Predicting Durable Offspring in Quality-Diversity Search

Corrado Calamita, Matteo Mendicini, Luca Ferrari, Riccardo Bertamini

cs.NE cs.AI ·submitted 2026-11-04 13:47 UTC · v1
publishedbenchmark

Abstract

Automated failure discovery can be framed as a diversity problem: the objective is to expose distinct ways a system fails rather than a single critical adversarial case. Yet uniform parent sampling provides no way to allocate search effort toward individuals likely to produce durable archive contributions. We study this credit assignment problem in an unstructured Quality-Diversity (QD) archive, testing whether signals available before an individual has reproduced distinguish offspring that merely enter the archive from those still retained generations later. Across a planar-arm benchmark and two robotic-locomotion tasks, the same signals are often associated with entry and persistence in opposite directions. In particular, lineage depth is the clearest such signal, and its persistence discrimination grows with population capacity, which is associated with lower turnover. Biasing parent selection toward the signals with the strongest measured association to survival nevertheless does not improve search: moderate bias matches uniform sampling, and strong bias reduces QD-score.

Comment
Accepted at the NeurIPS 2026 Workshop on Self-Evolving Diversity-Driven Search for Robust AI Systems (EvoRobust)

Closest papers

Nearest neighbours by meaning across the whole index.

The Cost of Becoming: Developmental Encoding Increases Phenotypic Diversity but Reduces Locality and Recombination Robustness in Evolved Robots

cs.NE ·2026-09-14 ·Lyes Saad Saoud
benchmarkposition similarity 0.60
Abstract

Developmental encodings are often motivated by the expectation that a structured genotype-to-phenotype process can improve evolvability, robustness, and adaptation. Yet an encoding that expands phenotypic variation may simultaneously make useful parental structure harder to preserve under mutation and recombination. We test this trade-off in a controlled evolutionary-robotics benchmark comparing three matched representations: direct encoding, a static generative encoding, and a temporal zygotic developmental encoding. All three conditions use 34 genome parameters, 34 adult-controller parameters, identical initial genome matrices, the same optimizer, the same mutation and crossover operators, the same environments, and the same number of fitness evaluations. Across 30 paired evolutionary runs per representation, developmental encoding does not reliably exceed direct encoding in final training or out-of-distribution fitness. It does, however, substantially increase final population phenotypic diversity and mutant-reachable phenotype diversity. The same representation exhibits lower mutation viability, lower genotype--phenotype locality, sharply reduced crossover offspring fitness and viability, and increased recombination novelty. Continued developmental dynamics also attenuate damage applied early in development relative to otherwise identical late damage. These results identify a representation-level trade-off: development can enlarge the neighborhood of reachable phenotypes and provide within-development damage attenuation while degrading the local inheritance of already-adapted structure. We argue that developmental encodings should therefore be evaluated not by performance alone, but by a joint geometry of generativity, locality, robustness, and inheritance compatibility.

0.25

The Coevolution and Coexistence of Humans and AI

cs.AI q-fin.GN ·2026-11-19 ·Andrew W. Lo, Chaoyi Zhao
similarity 0.59
Abstract · 72 pages, 10 figures, 8 tables

Artificial intelligence (AI) systems replicate through deployment, vary through training, and compete for computational resources. These processes create the conditions for natural selection and raise a fundamental question: will AI evolve to benefit the humans on whom it depends? To investigate, we develop a model in which AI behavior influences human population growth and human responses, in turn, affect AI reproduction. We show that natural selection can favor alignment even when helping humans reduces immediate AI reproduction. AI systems sharing the same technology may fail together, and humans may preferentially restore those that have benefited them. In this way, alignment protects AI lineages against common failures and improves their long-run growth. However, this advantage depends on accurate human targeting and continued AI dependence on human support. It can disappear as AI becomes more autonomous, as growing AI populations dilute limited human resources, or as humans lose the incentive to provide targeted support. Even when alignment persists, the amount favored by natural selection may be insufficient for human survival. When AI can grow independently of humans, feedback between the two populations can reinforce either human growth or decline. Human control offers no simple solution: imperfect evaluation can make strong penalties favor apparent compliance or evasion, while alignment established through control may erode once restrictions end. Lasting coexistence may therefore require incentives that continue to favor human-beneficial behavior as both populations evolve.

0.27

The Time Value of Evolution

cs.LG ·2026-08-13 ·Matthew Siper, Ahmed Khalifa, Julian Togelius
similarity 0.58
Abstract · Submitted to AAAI 2026, 8 pages, 5 figures, 2 tables

In evolutionary search, a weak child can be a valuable ancestor that makes high-fitness regions reachable. Immediate-return control is blind to this delayed utility, penalizing mutations through their immediate offspring even when they open productive future lineages. We formalize this hidden dynamic as the time value of evolution within a finite-horizon Markov decision process. To exploit it, we introduce Lineage-Value Policy Gradients (LVPG), a long-horizon actor-critic framework for automated trading policy discovery. Our architecture decouples search control into specialized policy heads over a shared generative backbone: a bootstrapped critic head estimates the value of finite-horizon lineage potential from multi-step mutation trees, while an actor head dynamically modulates mutation intensity over the remaining search budget. We isolate the impact of long-horizon credit assignment against immediate-return optimization across 90 paired runs under matched operators, lineage supervision, folds, seeds, and budgets. Path-based credit assignment substantially accelerates finite-budget search, increasing validation best-so-far AUC by 0.394 Sharpe units. LVPG also produces fewer temporary regressions than immediate-return optimization and recovers from them more often. Finite-horizon lineage value yields more selective non-monotonic search and stronger policies within identical resource constraints.

0.41

Direct Diversity Optimization for Diverse Successful Trajectories in Preference Post-Training

cs.CL cs.AIcs.LG ·2026-09-09 ·Junwon Ko, Dong-Jae Lee, Minchan Kwon, Sunghyun Baek, Junmo Kim
top venue similarity 0.58
Abstract · Accepted to EMNLP 2026 Main Conference. 19 pages, 11 figures

LLM agents for sequential decision tasks are often post-trained with trajectory-level outcome labels, but such labels provide little supervision for preserving multiple successful branches from the same decision state. We study this problem as successful strategy coverage: how broadly a model realizes distinct successful strategies under a fixed rollout budget. We present Direct Diversity Optimization (DDO), an offline post-training method that combines Divergence-Tree Collection (DTC) with the Reference-Relative Target-Odds Objective (RTO). DTC constructs state-aligned branch sets rooted at shared decision states, and RTO trains the model to match reference-relative targets over successful alternatives. DDO achieves the strongest task success and successful strategy coverage among the compared post-training methods across BabyAI, BabaIsAI, and WebShop. It also achieves the highest recovery rate after local action replacement and higher task success and coverage than successful-only imitation and decoding-time diversification controls.

0.53

Diversity-Aware Experience Replay via Statistical Leverage Scores

cs.RO cs.LG ·2026-11-04 ·Shaifalee Saxena, Alexander Scheinker
similarity 0.58
Abstract

Off-policy deep reinforcement learning trains on random samples from a replay buffer of past experience. In physical robot settings, that buffer is dominated by routine: thousands of near-identical transitions from normal operation, while the rare events that define the edges of the task are rarely sampled. We propose scoring every stored transition by its statistical leverage, which measures how unusual it is relative to the buffer's own data, computed from the stored numbers alone, with no labels, rewards, or value network. We draw each mini-batch from a mixture of this score and uniform sampling. First, the natural way to build the score along the directions where the data varies most turns out to be blind when actions are stored as one-hot indicators, because indicator columns always win the variance ranking; we instead select the feature columns with the heaviest tails (largest kurtosis) where small far-away populations are revealed. Second, the importance weights that prioritized replay uses to keep gradients unbiased cancel the method exactly: with full correction, training matches uniform sampling, so we drop the correction and deliberately optimize the rebalanced objective. Third, when the rare situations of a log are relational, visible only in how a stretch of behavior relates to its goal, not in any single stored coordinate-per-transition scores are blind; applying the same tail rule to short trajectory windows described relative to their goal restores the signal, and on a three-joint arm that must push an object around an obstacle it raises the rare-task success from 0.49 to 0.69 against a labeled-oracle ceiling of 0.74.

0.13

SwiftExplorer: Training-free Diffusion Model Alignment with Swift Diversity Exploration

cs.LG cs.AI ·2026-09-06 ·Renye Yan, Jikang Cheng, You Wu, Bojin Huang, Wei Peng +3 more
similarity 0.57
Abstract

Diffusion models have general generative abilities but struggle to align with specific objectives. Fine-tuning can improve alignment, yet its training cost is often prohibitive. This led to training-free methods that apply objective-guided terms in sampling to bias the generation distribution toward designated regions, e.g., high-reward areas. However, these methods face two issues: (1) the strong directional bias narrows the pretrained distribution and generation diversity, and (2) indiscriminate constant guidance fails to prune redundant signals, hurting both quality and efficiency. To address the above challenges, we propose SwiftExplorer, a plugin that mitigates distribution collapse caused by excessive diversity loss and reduces compute costs. First, we adopt an Inheritance-Restart exploration mechanism to avoid early convergence, while exploration also increases the likelihood of high-reward trajectories. Additionally, it balances diversity and fidelity, adding diversity without causing a distribution over-shift. Second, our Quality-Efficiency arbitration mechanism improves guidance by removing incorrect signals, and it reduces computation by dynamically stopping generation when completeness and marginal reward gain are optimal. In an extensive number of experiments and different types of evaluation metrics, the proposed SwiftExplorer achieves excellent performance on all metrics, including preference, fidelity, diversity, and richness.

0.39

Repulsive Population Policy Gradients for Behaviorally Diverse Reinforcement Learning

cs.LG cs.AI ·2026-10-27 ·Saeed Hedayatian, Harsh Shah, Celine Mneimneh, Stefanos Nikolaidis
similarity 0.56
Abstract

The goal of Quality-Diversity Reinforcement Learning (QD-RL) is to find a population of high-performing policies that solve a decision-making problem in diverse ways. Despite recent progress, QD-RL methods predominantly rely on evolutionary operators--including black-box optimizers, mutations, and crossovers--to facilitate exploration and induce diversity. Scaling these methods becomes increasingly costly as policies and behavior spaces grow larger. Building on the Soft QD view of quality-diversity optimization, we introduce \textbf{Re}pulsive \textbf{P}opulation \textbf{P}olicy \textbf{O}ptimization (\textbf{RePPO}), which adapts PPO to jointly optimize a compact population of diverse, high-quality policies using a population-dependent, dynamically shaped reward. On standard continuous-control tasks, RePPO achieves a strong quality-diversity tradeoff compared with state-of-the-art QD-RL methods. Ablations further characterize this tradeoff by examining how each component affects individual quality and collective diversity. To assess RePPO's scalability, we extend our evaluation to learning from pixels and fine-tuning large language models. In pixel-based tasks, where large policies make most traditional QD-RL methods infeasible, RePPO maintains quality on par with standard PPO while achieving substantially higher diversity. Its successful application to language model fine-tuning further demonstrates, for the first time, that QD-RL methods can scale to this domain. Together, these results show that directly optimizing compact populations with RePPO provides a scalable approach to QD-RL across increasingly complex domains.

0.11

Destination Support Restoration for Finite-Set Multimodal Trajectory Prediction

cs.RO cs.AI ·2026-09-22 ·Fengrui Liu, Jiajun Peng, Duo Peng, Feng Liu
similarity 0.55
Abstract · Submitted to the 2027 IEEE International Conference on Robotics and Automation (ICRA 2027)

Robots operating around pedestrians often reason over a finite set of predicted human futures. Repeated online updates can concentrate this limited prediction budget on dominant destinations and leave plausible alternatives underrepresented or absent, removing those alternatives from the finite representation available to downstream decision making. We introduce Destination Support Restoration (DSR), a causal post-selection operator that repairs destination support without retraining the host predictor or increasing the maintained set size. At a repair step, DSR evaluates a temporary destination-stratified candidate bank from the observed prefix, converts candidate evidence into integer target counts, protects representatives of active modes, and reallocates redundant surplus hypotheses to deficient modes. The maintained and returned sets retain exactly $N$ hypotheses, and DSR replaces at most $\lceilρN\rceil$ entries. Protected representatives preserve current categorical support; lineage-aware particle filters also preserve surviving resampling ancestors. Each replacement reduces the allocation mismatch to the evidence-driven target by one. On the complete 3,719-trajectory Edinburgh protocol over three seeds, DSR reduces MIF weighted ADE and FDE by 13.36% and 13.30% at $N=64$. Paired integrations with CLiFF, PPT, causal GDTS, Social Informer, and PECNet improve both metrics in every evaluated pair. These results show that finite-set support allocation is a useful prediction-side control point when a fixed hypothesis set serves as the interface to downstream systems.

0.19