CS Paper Radar

From Search to Signal: Search-Value Credit for Evolutionary Automatic Heuristic Design

Yilun Yuan, Tianyu Zhou, Zhenzhou Tang

cs.LG cs.AIcs.NE ·submitted 2026-09-30 09:33 UTC · v2

Abstract

Large language models generate candidate programs for automatic heuristic design, while evolutionary search evaluates and selects them to guide further generation. Online post-training uses rewards from candidate evaluations to adapt the generator. Retained programs can also guide later generation as parents. This motivates rewards that reflect their reuse opportunities beyond immediate fitness. We propose Search-value credit (SVC), which constructs response-level rewards from actual admission to the retained set and a short-term parent-exposure proxy. Admission determines reward eligibility; the proxy weights eligible responses by reuse opportunities under existing parent-selection rules. Rewards are computed after candidate evaluation, without evaluating descendants. SVC retains the search and learning procedures of the Co-Evolution of Algorithms and Language Model (CALM) framework. Across four combinatorial optimization tasks with the same base model and response budget, SVC achieves the best mean final search performance on three tasks among six conditions. Under identical prompts and parents on traveling salesman and capacitated vehicle routing tasks, SVC-trained generators yield a higher mean proportion of responses whose programs outperform the best prompting parent than those trained with CALM's native reward. These findings support reward construction using search retention and selection information.

Comment
80 pages, including supplementary material. Substantially revised version introducing Search-Value Credit (SVC), with a redesigned reward mechanism and expanded experiments

Closest papers

Nearest neighbours by meaning across the whole index.

Multi-Task Evolution for Zero-Shot Cross-Problem Generalization using LLMs

cs.AI ·2026-10-02 ·Zhuoliang Xie, Changliang Zhou, Genghui Li, Zhenkun Wang
similarity 0.70
Abstract

Designing effective heuristics for diverse combinatorial optimization problems requires substantial expertise and repeated search. Large language models (LLMs) automate heuristic generation and refinement, but heuristic search typically depends on evaluation feedback from the problem being optimized. Generalizing to new problem definitions using only source-task feedback therefore remains a central challenge. We introduce MECo, an LLM-driven multi-task evolutionary framework for zero-shot cross-problem generalization. MECo maintains task-conditioned heuristic populations and uses a transfer gap based on cross-task population performance to guide their interactions. These interactions enable the transfer and recombination of heuristics. A complementary selection criterion then constructs a compact heuristic set by rewarding each member's additional coverage of source combinations. The selected set is applied to target problems without further search or adaptation. Experiments on 32 problem variants across vehicle routing (VRP) and flexible job-shop scheduling (FJSP) show that MECo achieves the lowest mean costs compared with eight automated heuristic design (AHD) baselines under the same budgets. On out-of-domain problems, it outperforms the strongest baseline in each family. Moreover, integrating the framework of MECo with different AHD methods improves their ID and OOD performance in both families, supporting its effectiveness across different methods.

0.20

LLM-Driven Joint Evolution of Coupled Heuristics Components for Routing Optimization

cs.NE ·2026-09-02 ·Juntao Wei, Yangming Zhou, Zhibin Jiang, Shan Jiang
similarity 0.69
Abstract

Heuristic design for combinatorial optimization remains heavily reliant on expert knowledge, while existing large language model (LLM)-enhanced evolutionary methods typically evolve isolated algorithmic components, even when one determines the search state on which another operates. This paper proposes LLM-driven Heuristic Components Joint Generation (LLM-HCJG), a population-based framework that jointly generates and co-evolves interdependent heuristic components under a shared design blueprint. Applied to guided local search (GLS), LLM-HCJG couples solution initialization with penalty construction and embeds the generated pair into an enhanced online search mechanism. The resulting form is further transferred from the traveling salesman problem (TSP) to the capacitated vehicle routing problem (CVRP). Theoretical analysis establishes the non-separable state-transition effects between the two components and the advantage in generation consistency. Across synthetic instances and 41 public TSPLIB/CVRPLIB benchmarks, LLM-HCJG attains consistently low optimality gaps, including best or tied-best results on 28 of 29 TSPLIB instances and all 12 CVRPLIB instances. Ablation and structural analyses further indicate that these gains are associated with cross-component compatibility and alignment rather than isolated-component recombination. These results support effective cross-instance transfer within the evaluated routing settings under limited-sample, modest-cost training.

0.31

EPOCH: Coverage-Aware Evolution with Experiential Preference Optimization for LLM-based Automated Heuristic Design

cs.AI cs.CL ·2026-10-12 ·Oguzhan Gungordu, Ayse Tursucular, Siheng Xiong, Faramarz Fekri
code/data similarity 0.67
Abstract

LLM-based automated heuristic design (AHD) discovers heuristics for combinatorial optimization problems (COPs), where an LLM generates candidates in an evolutionary search. Existing frameworks freeze the generator or fine-tune it on a scalar fitness signal hiding which instances each heuristic covers. Reflection-based frameworks improve heuristics with natural-language feedback, but it stays in the prompt and never reaches the generator's weights. We propose EPOCH, which makes instance coverage the objective of the evolutionary search and fine-tunes the generator to internalize reflection. Parent selection, operator allocation, and stagnation recovery all optimize how well the heuristics jointly cover the training instances, so the search returns a complementary heuristic ensemble. The generator learns from its own search by experiential preference optimization. A teacher LLM returns a natural-language insight on the generated heuristics, and the generator produces new heuristics with this insight. These improvements join the preference pairs for direct preference optimization (DPO). Since DPO only teaches the generator to rank these improvements, a reflective self-distillation (RSD) term makes it more likely to generate them on its own, and the gains accumulate over the long horizon of the search. Across COPs and open-source LLM backbones, EPOCH's heuristic ensembles outperform inference-only and fine-tuned baselines and generalize across problem scales.

0.23

$R^2E:$ A Role-driven Reward Evolutionary Framework for Automated Reward Function Design

cs.RO ·2026-10-23 ·Shouhao Chang, Xuan Liu, Hongye Zhu, Xinning Chen, Shigeng Zhang
similarity 0.66
Abstract

Designing effective reward functions is a critical challenge in Reinforcement Learning (RL), which traditionally requires costly manual trial-and-error. While Large Language Model (LLM)-based methods have shown promise in reward design, undifferentiated sampling and limited use of historical feedback often lead to insufficient exploration of the reward space and optimization instability. To address these challenges, we propose $R^2E$, a Role-driven Reward Evolutionary framework for automated reward function design that structures the process through explicit role specialization. $R^2E$ employs multiple LLM roles with complementary objectives: an Explorer that promotes novelty to expand the search space, a Guardian that performs conservative refinements to improve training stability, and an Evolver that recombines high-performing rewards via crossover and mutation to integrate effective structures. A dynamic scheduling strategy coordinates these roles, progressively shifting the search from broad exploration to focused exploitation. To mitigate optimization instability and provide a medium for cross-role collaboration, $R^2E$ incorporates a Global Elite Pool that retains the best-performing rewards to guide subsequent generations, thereby enabling coordinated interaction among roles and ensuring a reliable and consistent refinement process. Extensive experiments on multiple robotic tasks demonstrate the effectiveness of our method.

0.18

Directing to Diversify: Multi-Directional Search for Automatic Heuristic Design

cs.AI ·2026-10-22 ·Jianghan Zhu, Jianan Zhou, Fei Liu, Ni Zhang, Yaoxin Wu +2 more
similarity 0.65
Abstract · 46 pages, 9 figures, 19 tables. The main paper, including references, is 13 pages; appendices are included

Large language models (LLMs) have demonstrated strong potential to automate heuristic design for NP-hard combinatorial optimization (CO) tasks without human intervention. Recent methods predominantly rely on evolutionary computation (EC) frameworks, where heuristic diversification is mainly induced through EC-like operators or downstream selection, while the direction of heuristic transformation remains largely unharnessed. In this paper, we propose a multi-directional Perturbation-Rewrite Search (PRS) framework for automatic heuristic design that factorizes heuristic transformation through directional search primitives at the generation level. Specifically, PRS operates on heuristic ideas through three complementary directions: expansion toward richer and more expressive designs, compression toward simpler and less specialized designs, and steering for fine-grained refinement, together enabling broader and more sustained exploration of diverse regions in the heuristic design space. PRS further organizes candidate exploration within a breadth-first search-based process with adaptive operator control, which ensures thorough candidate utilization and sustained exploration, and maintains strong search dynamics while preventing premature convergence. Extensive empirical results across diverse CO tasks show that PRS consistently discovers high-quality heuristics, demonstrating the effectiveness and generality of explicitly directional exploration design.

0.24

Optimize Cheap, Deploy Strong: Cost-Aware Cross-Tier Transfer for Evolutionary Optimization

cs.LG cs.AIcs.CLcs.NE ·2026-08-11 ·Tal Oved, Roi Pony, Oshri Naparstek, Udi barzelay
similarity 0.65
Abstract

Evolutionary optimization of LLM prompts and agentic programs (e.g., GEPA) is dominated by fitness evaluation: scoring each candidate runs an answering LLM over a validation set, so the evaluator's price tier dictates total search cost. We restructure that search by decoupling the three roles an LLM plays, running the high-volume answering role on the cheapest tier, reserving a strong model for the rare reflection/variation operator, then exploiting upward cross-tier transfer to deploy the cheaply evolved prompt on a stronger target. We contribute a cost-controlled characterization of when cheap-tier search substitutes for target-tier search, and where it fails. Across four tasks (HotpotQA, IFBench, LiveBench-Math, HoVer) and eleven models in four model families, the resulting prompt matches or exceeds same-tier optimization while placing over 96% of search tokens on the cheapest tier, at 5.6-14x lower search cost, rising to 25-54x where reasoning tiers emit long chains of thought on every fitness call.

0.04

Fusion is the New Mutation: Bandit-Guided Evolution on Workflow Graphs

cs.AI ·2026-10-04 ·Zhiwei Shang, Jiahang Sun, Mingrong Gong, Mingze Kong, Zikun Qu +7 more
similarity 0.64
Abstract

Automated agentic workflow optimization relies on costly evaluations, making it essential to allocate a limited evaluation budget effectively. Multi-parent fusion can reuse designs from previously discovered workflows, but identifying promising parent combinations requires learning from limited fusion feedback. We introduce DAGO (Directed Acyclic Graph Optimization), a contextual-bandit-guided framework that learns which parent workflows to fuse under a limited evaluation budget. DAGO formulates each candidate parent combination as an arm, represented by pretrained embeddings of its constituent workflows' code and prompts. A diagonal LinUCB policy learns a shared reward model across arms and balances exploitation of arms with high predicted offspring quality against uncertainty-driven exploration. After an arm is selected, an LLM generates a child workflow through summary-guided fusion, and the child's validation score serves as the reward for updating the bandit. A shared directed acyclic graph maintains discovered workflows and their multi-parent lineage, providing an expanding pool of parents for subsequent arm proposals. Across six benchmarks covering mathematical reasoning, code generation, and question answering, DAGO achieves the highest macro-average score among the evaluated baselines. Under matched validation-evaluation budgets, it improves over AFlow from 80.3 to 81.7 while reducing aggregate search expenditure by 11.2%. Ablation studies show that LinUCB-guided arm selection outperforms both random selection and its exploration-free variant, supporting the value of feedback-driven selection and exploration-exploitation balance.

0.23

Explore More, Validate Less: Efficient Reflective Prompt Optimization

cs.CL ·2026-10-19 ·Xiaofan Zhou, Lu Cheng
similarity 0.63
Abstract · 10 pages

Reflective prompt optimization improves instructions using task feedback, but coupling candidate generation with full-validation-based selection can make search costly and sequential. We propose a two-stage framework that decouples reflective candidate-pool construction from validation allocation. During construction, sampled parents first answer training questions, and groups of their saved responses guide parallel reflection and revision. Each child is screened on its group's questions against the corresponding parent scores. Improving, nonduplicate prompts enter the pool and can become parents without full validation. After construction, an exploration policy selects candidates from the frozen pool under a fixed full-validation budget, and the prompt with the highest measured validation score is returned. This separation allows candidate-pool size and validation budget to be set independently. We evaluate random selection and a multilayer perceptron with similarity-based exploration (MLP + similarity) on five benchmarks using Qwen3.5-9B and Llama-3.1-8B-Instruct. With 100 retained candidates and 20 full validations, our framework improves over the initial solver by up to 11.22 percentage points. MLP + similarity improves over the initial solver in all ten comparisons, although neither selection policy consistently outperforms the other. These results support separating reflective search from budgeted validation while leaving room for stronger candidate-selection policies.

0.40