CS Paper Radar

Semantics-Guided Automatic Tensorization for Multiobjective Evolutionary Algorithms: A Multi-Agent Framework

Zhenyu Liang, Beichen Huang, Bowen Zheng, Ran Cheng

cs.NE ·submitted 2026-09-02 10:00 UTC · v1
benchmark

Abstract

Multiobjective evolutionary algorithms (MOEAs) naturally expose population-level parallelism, but many mature implementations encode their computation in sequential program structures designed for central processing units. Exploiting modern tensor computing platforms therefore requires more than direct code translation: the implementation must be restructured without changing the defining optimization mechanism of the underlying MOEA. We formulate automatic tensorization for MOEAs as semantics-guided computational restructuring and develop Evolutionary Code Conversion (EvoCoCo), a multi-agent framework that realizes this formulation. EvoCoCo reconstructs algorithm-specific states, dependencies, operators, and update logic into a structured semantic representation and organizes them through a shared tensorization blueprint. Specialized transformation branches then explore alternative tensor realizations, while execution feedback guides validation, repair, and candidate selection. Experiments on a benchmark of 48 MOEAs evaluate migration reliability, optimization fidelity, and computational scalability. Under matched large language model backends, EvoCoCo attains higher migration reliability than direct one-shot translation. Across the benchmark suites, 88.2% of valid comparisons satisfy the predefined optimization-fidelity criterion. The tensorized implementations also exhibit increasing acceleration on graphics processing units as population size or decision dimension grows, with median measured speedups ranging from $22.6\times$ under population scaling to $80.2\times$ under decision-dimension scaling. External-source and ablation studies further assess transfer beyond the main benchmark and the roles of the major framework components.

Closest papers

Nearest neighbours by meaning across the whole index.

The Art of Closed-Formula Defaults: Search-Free Code Generation for Tensor Operators

cs.MS ·2026-09-10 ·Paolo D'Alberto, Ashish Sirasao
similarity 0.64
Abstract · 12 pages, 2 Figures, 3 tables

Agentic search and automated optimization of GPU kernels are powerful tools for large language model inference. Their effectiveness, however, depends not on the sophistication of the search itself, but on the clarity of the optimization problem being solved. We provide an application-first approach that drives a hierarchical code generation tool from operator specifi cation down to GPU instructions, and show that a clearly defined computational model makes the optimization problem tractable.

0.22

Online Automated Algorithm Design with Large Language Models

cs.NE ·2026-09-21 ·Zhiyao Zhang, Yichen Li, Xingyu Wu, Liang Feng, Kay Chen Tan
benchmark similarity 0.61
Abstract

Large language models (LLMs) enable automated algorithm design (AAD) through reasoning and code synthesis. However, most existing LLM-based AAD methods separate algorithm design from target optimization, deploying a fixed design even as the optimization state evolves. Conventional adaptive optimizers can respond to such changes, but their adjustments remain confined to predefined parameters, operators, or strategies. To address these limitations, we introduce online LLM-based AAD, a novel optimization paradigm that treats the algorithm itself as a state-dependent decision variable. At each stage, LLM agents synthesize an algorithm with new behavior logic from the current optimization state. Executing the generated algorithm advances the search and provides feedback for subsequent designs, coupling algorithm design with target optimization without requiring a separate offline algorithm pretraining stage. To implement this paradigm, we propose OnDesign, a multi-agent framework that reconciles competing design perspectives to synthesize executable algorithms and uses execution feedback to refine how runtime evidence is interpreted for subsequent designs. We evaluate OnDesign across two mainstream black-box optimization paradigms on three scenarios: Bayesian optimization, evolutionary continuous optimization, and evolutionary mixed-variable optimization. Extensive experiments on six benchmark suites and one engineering problem across multiple problem dimensions demonstrate superior overall performance over conventional optimizers and offline LLM-based AAD methods.

0.15

AscendMOEA: An Ascend NPU-Accelerated Platform for Tensorized Multi-Objective Evolutionary Algorithms

cs.NE ·2026-11-09 ·Qianlong Dang, Zhiyang Zhang, Baosheng Li, Tianqing Zhu, Junhu Ruan
benchmarkcode/data similarity 0.61
Abstract

Multi-objective evolutionary optimization increasingly involves large populations, expensive problem evaluations, and diverse algorithmic structures. These workloads expose the limited parallelism of loop-based implementations, while rewrites for accelerator execution can alter selection order, tie handling, and population updates. AscendMOEA is presented as a scalable platform that executes multi-objective evolutionary algorithms (MOEAs) on Huawei Ascend neural processing units (NPUs) and conventional central processing units (CPUs). The platform represents the decision vectors, objectives, constraints, and auxiliary states as tensors and provides reusable interfaces for algorithms, problems, population operators, quality indicators, monitoring, and repeated experiments. Dense computations are executed in batches, such as dominance, distance, variation, and problem-evaluation calculations. Short recurrences with causal dependencies retain their prescribed order and consume precomputed tensor results, which preserves the behavior of the original algorithms. AscendMOEA contains 18 algorithms, 155 benchmark and application problems, 19 public operator interfaces, and 11 quality indicators. Experiments on eight Ascend NPUs and 192 physical CPU cores evaluate operator throughput, end-to-end runtime, memory consumption, numerical agreement, and implementation ablation. Tensorized operators obtain a median acceleration of 1.58 times on the NPU. At large population scales, complete algorithms achieve a median acceleration of 5.76 times while retaining the recorded quality indicators. Moreover, the results identify the workload scale and ordered computation that determine whether accelerator execution is beneficial. AscendMOEA is open-sourced at https://github.com/dqlme/AscendMOEA.

0.49

Harnessing Language Models as Optimization Modeling Agents

cs.AI ·2026-10-22 ·Yunxiang Luo, Xin Xu, Tianhao Chen, Zhixi Xiong, Peiqi Jiang +5 more
similarity 0.60
Abstract

Large language models (LLMs) can translate natural-language optimization problems into solver programs, but reliably producing correct formulations and implementations remains challenging. Existing approaches improve optimization modeling through agentic workflows or task-specific post-training. This raises a practical question: can a lightweight harness using a single model role achieve competitive performance without additional training? We introduce OPTICYCLE and OPTISTATE, two lightweight inference harnesses for optimization modeling, and evaluate them on nine benchmarks comprising 2,634 problems using 11 Qwen variants spanning 0.6B-27B parameters. Our results show that execution-guided revision consistently improves macro accuracy over direct generation for all evaluated models, with gains of up to 49.46 percentage points. Despite their simplicity, both harnesses outperform six adapted agentic workflows in same-model comparisons and four evaluated post-trained optimization models at comparable model sizes, without additional training. Explicit formulation state provides further but model-dependent improvements, suggesting that additional structure is not universally beneficial.

0.16

Meta-Reflective Evolution Effectively Improves Production AI Systems

cs.LG cs.AIcs.CLcs.SE ·2026-10-08 ·Seung Won Wilson Yoo, Chang Zeng, Aditi Somayajula, Sahithi Ankireddy, Rohan Kosalge +5 more
benchmark similarity 0.59
Abstract

Large Language Model (LLM)-guided evolutionary optimization has successfully improved prompts, programs, and even entire AI systems. Most optimization methods select a candidate from the search frontier, then mutate it in one LLM pass based only on its failures. This focus on the frontier can stall on production AI systems, where isolated failures reveal little about complex, hidden interactions. We introduce CoCoEvolve (Cortex Code Evolve), an evolutionary optimizer built around what we call meta-reflection, wherein an agentic mutator (Cortex Code) selectively retrieves evidence from the entire evolutionary history and probes the live system before verifying and proposing mutations. We apply CoCoEvolve to optimize three classes of production AI systems: data agents, LLM-powered data pipelines, and data engineering workflows. On DABStep, CoCoEvolve tops the public leaderboard, reaching 92.8% held-out hard accuracy compared to 78.4% for Meta-Harness and 30.4% for GEPA under matched conditions. Across six LLM data pipeline workloads, it outperforms every evaluated baseline. On data-eng-bench, CoCoEvolve lifts test-set solve rate from 59.1% to 77.4%, past the best baseline of 73.6%. It also has 11 percentage points higher accuracy than GEPA on HotpotQA while remaining competitive with specialized optimizers on established mathematical and systems benchmarks. Matched ablations confirm that each component of CoCoEvolve contributes to these gains, demonstrating the value of meta-reflection for optimizing production AI systems.

0.18

OptiMAS: Automatically Optimize Multi-Agent System

cs.MA ·2026-08-22 ·Yuxin Cheng, Chang Liu, Hanxin Yu, Haochen Tan, Taiqiang Wu +7 more
top venue similarity 0.59
Abstract · EMNLP 2026

Automated evolution of Multi-Agent Systems (MAS) holds significant potential for reducing the manual effort required to design and optimize LLM-based agent architectures. However, extant search-based paradigms face a fundamental trade-off, where an expanded optimization scope exacerbates evolutionary instability, while discrete branch-and-discard search isolates insights across lineages. To address these limitations, we propose a continuous, data-driven optimization paradigm built upon a unified ReAct-based infrastructure that reconciles a broad optimization scope with operational stability. Under this paradigm, we present OptiMAS, a task-agnostic agentic optimizer that leverages textual interaction trajectories and task feedback as loss signals for end-to-end MAS evolution. Equipped with a novel dual-track memory mechanism, OptiMAS sustains performance improvement over extended optimization horizons. Evaluation on four heterogeneous agentic benchmarks with three varying scale and accessibility LLM backbones, demonstrates that OptiMAS consistently achieves competitive or superior accuracy relative to both domain-specialized hand-crafted systems and existing evolutionary methods. Our work establishes a practical milestone toward robust, automated MAS evolution.

0.30

TGR-EvoDAG: A DAG-based Multi-Agent Code Evolution Framework for Generative Recommendation on Tencent

cs.AI ·2026-10-23 ·Junlin He, Junzhang Zhu, Yudong Li, Yihong Tang, Xuefeng Jiang +3 more
similarity 0.58
Abstract

Large Language Model (LLM) agents have enabled automated code evolution for algorithm discovery. However, current recursive self-improvement (RSI) research for sequential recommendation faces a dilemma: general-purpose frameworks lack domain-specific feedback, while recommender-specific frameworks rely on linear evolution. Crucially, naively integrating them fails when applied to Generative Recommendation (GR). Unlike end-to-end models, GR relies on a module-cascaded pipeline (tokenization, pre-training, and alignment). In this paradigm, existing evolution methods suffer from: i) cost blindness, squandering budgets by ignoring stage-asymmetric retraining overheads; and ii) short-sighted evaluation, discarding promising upstream innovations that temporarily regress metrics before downstream adaptation. To bridge these gaps, we propose TGR-EvoDAG, a multi-agent code evolution framework tailored for GR. TGR-EvoDAG models code evolution as a Directed Acyclic Graph (DAG) to maintain multiple evolution branches and merge complementary innovations, unifying advanced search with rich GR diagnostic feedback. We further introduce stage-aware cost planning to optimize GPU-hour budgets, and active downstream co-adaptation to navigate temporary regressions and unlock cross-stage gains. On public Amazon datasets, TGR-EvoDAG enables the first autonomous evolution of a complete GR pipeline, improving offline metrics by up to 37.4% and Agent4Rec simulated user satisfaction by 19.7% over the strongest baseline. Furthermore, in industrial deployment on Tencent's large-scale commercial media platform, the evolved GR achieves a 12.16% relative improvement in offline Hit@1 and a statistically significant 0.40% increase in online click-through rate (CTR).

0.21

Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability

cs.LG cs.AI ·2026-08-31 ·Matvei Tarasov, Salman Ahmadi-Asl, Andre L. F. de Almeida, Andrzej Cichocki
surveycode/data similarity 0.58
Abstract

Large language models (LLMs) are built from structured high-dimensional objects such as token representations, weights, adaptation updates, caches, and activations, whose multilinear structure is underexploited by the conventional matrix-centric view. Tensor decompositions and tensor networks provide a principled algebraic language for this structure, yet the literature often treats them as isolated compression mechanisms. This survey organizes tensor methods for LLMs through two complementary views: a seven-stage lifecycle taxonomy covering tokenization, embeddings, pre-training, adaptation, compression, inference, and interpretability, and a component view covering embeddings, attention, and feed-forward networks. We provide unified notation and theoretical foundations, analyze tensorization strategies for individual Transformer components, and compare methods at each lifecycle stage while making differences in evaluation protocols and model scales explicit. We further connect tensor methods to neighboring efficiency techniques and probabilistic tensor networks. Finally, we synthesize open challenges and introduce $ρ_{\rm gap}$, a metric for the compression-realization gap between theoretical memory reduction and measured system-level speedup. By treating tensorization as a common structural principle, the survey provides a structured entry point to tensorized language models and clarifies when parameter savings can plausibly translate into memory efficiency, computational efficiency, or interpretability. The GitHub page dedicated to this paper is accessible at \href{https://github.com/ma-tt-a/awesome-tensor-methods-for-llms}{this https URL}.

0.54