CS Paper Radar

Multitask Interactions in Recurrent Dynamics across 100 Cognitive Tasks

Sílvia García-Hernández, Leema Hamid, Manuel Brenner

cs.LG cs.AIq-bio.NC ·submitted 2026-10-24 21:38 UTC · v1

Abstract

The brain continually learns tasks with very different computational demands. Recurrent neural networks (RNNs) have been used as dynamical surrogates for studying task solutions, with recent work extending this approach to joint training on multitask batteries. However, it remains unclear how tasks interact as these batteries grow in scale and heterogeneity: under what conditions do they transfer or interfere with one another, and how do these effects relate to task complexity and model structure? We study this question by simultaneously training hierarchical Almost-Linear RNNs (AL-RNNs) on 100 tasks spanning a wide range of cognitive demands. AL-RNNs allow us to vary the amount of nonlinearity in the recurrence, while hierarchical task features provide a low-dimensional interface for task-specific adaptation within shared dynamics. On this task battery, performance improves with both nonlinear capacity and task-feature dimensionality, with the joint hierarchical model outperforming both individually trained models and a single unstructured joint model. Many tasks are individually solvable with linear dynamics, yet cannot be solved together by a shared linear model. Through hierarchical task features, task-specific linear region usage, and task geometry, the trained model provides several related but distinct measures of task similarity that capture how computations are separated and combined within the shared backbone. Across a series of experiments, we relate these representations to individual task complexity, joint solvability, pairwise transfer learning, and leave-one-out adaptation. Together, our experiments characterize task interactions in multitask recurrent learning while demonstrating both the potential and the challenges of reusable dynamical backbones for cognitive computation.

Closest papers

Nearest neighbours by meaning across the whole index.

NOCE-Net: Representation Learning for Battery Operational Context via Nested Sequence Modelling

cs.CE eess.SY ·2026-10-15 ·Arnab Bhattacharjee, Wayes Tushar, Tapan Kumar Saha
top venuecode/data similarity 0.64
Abstract · 38 pages, Accepted in NeurIPS 2026

Learning effective representations from battery operational history is essential for predictive health management and control-oriented applications. Existing approaches rely on response-dependent engineered features or short early-life windows, limiting generalisability and precluding use in prospective settings where future responses are unavailable. We propose NOCE-Net, a deep nested sequence model that learns control-enabling representations directly from raw cyclic operational or telemetric data. Our data pipeline, re-BatteryData, standardises heterogeneous aging datasets into semantically equivalent Full Equivalent Cycles (FECs) defined by absolute charge throughput, enabling end-to-end learning across diverse operational regimes. NOCE-Net combines multi-patch transformer encoders for intra-cycle dynamics with a GRU backbone for inter-cycle state evolution, operating on approximately $700$ batteries across ten public datasets spanning over $300$ operational profiles. In zero-shot evaluation on unseen battery chemistries and usage patterns, NOCE-Net achieves $13.7\%$ MAPE in voltage response prediction, outperforming strong long-sequence baselines by over $43\%$ while using $40 \times$ fewer parameters. Shape and component diagnostics of the learned hidden states reveal that the GRU's principal direction of evolution achieves Spearman $|ρ| \geq 0.95$ with the capacity fade trajectory on average across all datasets with monotone degradation profiles, including zero-shot test batteries, providing evidence that the representations encode degradation dynamics transferable to downstream tasks beyond battery response modelling. Code is available at https://github.com/BeArnab96/NOCE-Net.

0.39

Learning Behaviorally Relevant Dynamics in Multimodal Neural Recordings

cs.AI cs.LGcs.NE ·2026-11-03 ·Parastoo Azizeddin, Eray Erturk, Maryam M. Shanechi
similarity 0.64
Abstract

Multimodal neural data can enable a more complete understanding of brain dynamics underlying behavior. Modalities such as spiking activity, local field potentials (LFPs), and behavior capture diverse spatiotemporal aspects of brain processes, yet prior neural-behavioral models that separate sources of variability rely on a single neural modality. By leveraging these complementary strengths, multimodal neural fusion can provide a unified, rich representation of brain-behavior processes and address the limitations of single-modality analyses. Here we develop BREM-NET, a nonlinear multimodal dynamical model that integrates behavior with multiple neural modalities having distinct statistical characteristics and temporal resolutions. BREM-NET performs multimodal neural fusion during inference while separating behaviorally relevant and residual neural dynamics via a structured learning framework. In a public multimodal non-human primate dataset with simultaneous spiking and local field potential recordings during a 2D reaching task, BREM-NET separates these two types of dynamics and learns them more accurately, as reflected in better behavior decoding, better neural prediction, and stronger latent alignment with behavior, and it retains these capabilities when neural modalities are asynchronous or suffer dropouts, which are major challenges in real-world recordings. This framework provides a new tool for studying behaviorally relevant neural computations across spatiotemporal scales of brain activity.

0.25

HorizON: A Multi-Horizon Deep Operator Network for Property Prediction of Organic Battery Stacks

cs.LG cond-mat.mtrl-sci ·2026-10-27 ·Subhash V. S. Ganti, Lukas Woelfel, Arla Diengdoh, Tzvetan Mihaylov, Thomas Nevolianis +2 more
similarity 0.63
Abstract

Transition-metal-based batteries rely on critical elements such as lithium, cobalt, and nickel. Their extraction creates environmental and supply-chain challenges. Organic electrode materials (OEMs) reduce this reliance and provide broad molecular tunability. However, OEM-based batteries still show limited long-term cycling stability. Their design also requires the joint optimization of coupled metrics such as voltage, specific capacity, capacity retention, and energy density. These metrics depend on the full battery stack and on the interactions among its components rather than on the active material alone. In our previous work, we developed a data-fusion multi-task framework for average discharge voltage and first-cycle specific capacity. Both targets are single-valued. Here, we extend that framework to multi-horizon prediction by predicting a property at every cycle of a discharge trajectory rather than at a single cycle. Our multi-fidelity and Multi-horizon Deep Operator Network learns DFT redox potentials and cycle-dependent specific-capacity trajectories within one model. We train it on complete battery-stack representations and usage conditions, so it learns battery-stack interactions rather than cathode-only effects. This framework supports long-term property prediction, inverse design, and process optimization for organic battery systems.

0.15

From Redundancy to Minimality: Fixed-Point-Guided Hierarchical Reduction of Learned Piecewise-Linear Dynamics

cs.LG nlin.CD ·2026-10-01 ·Hiroto Tamura, Gouhei Tanaka
similarity 0.63
Abstract · 27 pages, 6 figures

Understanding a nonlinear dynamical system from time series requires not only reproducing its trajectories, but also identifying a simple representation that preserves its essential dynamical structure. Almost-linear recurrent neural networks (AL-RNNs) are piecewise-linear RNNs in which only a subset of units use ReLU nonlinearities, so that nonlinear capacity is explicitly controlled by the number of ReLU units. Their activation patterns define linear regions, represented as symbols, whose observed transitions form a symbolic transition graph. However, directly training AL-RNNs with few ReLU units to realize minimal dynamical representations can be unreliable. We ask whether an AL-RNN with more ReLU units can instead be trained first and systematically reduced to a minimal dynamical representation. We introduce a fixed-point-guided hierarchical reduction procedure that progressively linearizes selected ReLU units, merging neighboring linear regions and graph nodes while preserving distinct symbols containing fixed points (FPs). The resulting reduction tree defines a hierarchy of progressively simpler candidates. Each reduced candidate is initialized from the parent parameters and retrained under guidance from the parent dynamics. We also prove that reproducing $Q$ distinct fixed points requires at least $Q$ FP-containing symbols, providing a certificate of symbol-level minimality when this bound is attained. On the 3-scroll Chua system, direct training with the theoretical minimum of three ReLU units achieves high-fidelity minimal realizations in only 20% of seeds, whereas our learn-reduce-retrain strategy increases the seed-macro success rate to approximately 71% at the same final nonlinear capacity. These results show that redundant nonlinear capacity can serve as a scaffold for discovering and realizing minimal dynamical representations.

0.34

Continual Learning of Dynamical Systems in Recurrent Neural Networks through Recyclable Unit Gating

cs.LG ·2026-09-29 ·Sima Hashemi, Daniel Durstewitz, Georgia Koppe
benchmark similarity 0.61
Abstract · 34 pages (including Appendix), 15 Tables, 7 Figures

Dynamical Systems Reconstruction (DSR) aims to infer models from observed time series that reproduce a system's qualitative long-term behavior. Continual DSR (cDSR) requires learning new systems while preserving previously learned dynamics, yet even small parameter updates in recurrent models can qualitatively alter their behavior over long autonomous rollouts. We benchmark established continual learning (CL) methods spanning parameter regularization, replay, and parameter isolation on the fully trainable and interpretable Almost-Linear RNN (AL-RNN). Parameter isolation preserves earlier dynamics most effectively, but excessive task-specific allocations can rapidly exhaust a fixed-size network. We therefore introduce Continually-Recyclable Unit-Gating (CRUG), which conserves capacity through compact allocation and forward transfer. Differentiable gates trained with an $L_0$-based penalty select task-specific units, while unused units are recycled for subsequent tasks. Directed connections allow later tasks to reuse earlier representations without affecting the dynamics of previously committed units. CRUG achieves the strongest reconstruction--capacity trade-off among the tested methods with zero forgetting and reliably learns a heterogeneous sequence of nonlinear and chaotic systems. Furthermore, we show that forward transfer is more pronounced and useful when tasks share similar underlying dynamics. Lastly, we demonstrate that CRUG's advantages extend beyond autonomous cDSR to sequential cognitive tasks.

0.24

Continuous Memory Machines

cs.AI ·2026-10-06 ·Ciaran Regan, Kai Arulkumaran, Luke Darlow, Stefania Druga, Sebastian Risi +1 more
publishedcode/data similarity 0.61
Abstract · NeurIPS 2026 Workshop: Personalized, Aligned, Long-Term Memory for AI Systems (PALM)

Recurrent neural networks typically compress information into a single vector-valued recurrent state, forcing short-term computation and long-term retention to share the same representation. Past extensions alleviate this bottleneck by increasing the memory capacity or separating timescales, but lack the combination of rapid neuron-level processing and longer-term retention found in biology. To that end, we introduce the Continuous Memory Machine (CMM), a recurrent architecture with matrix-valued short- and long-term memory states serving distinct functional roles. Building on the Continuous Thought Machine (CTM), the CMM's short-term memory tracks recent neural activity, with uniquely parameterized neuron-level models learning to use these activity patterns for computation. A persistent long-term memory stores information for later use, with a Transformer jointly updating both memory stores, providing an expressive bidirectional read--write mechanism such that each store can reorganize its own contents and both read from and write to the other. Across algorithmic, in-context learning, and recurrent reasoning tasks, the CMM outperforms a broad suite of baselines, exhibiting stronger generalization than prior memory-augmented networks while preserving the CTM's interpretable attention patterns. Code is available at https://github.com/SakanaAI/continuous-memory-machines.

0.43

TimeBraid: Unifying Time Series and Language for Understanding and Forecasting

cs.CL cs.AIcs.CE ·2026-09-24 ·Xinyue Wang, Jiacheng Pang, Kun Zhou, Kexin Zhang, Defu Cao +5 more
similarity 0.61
Abstract · 57 pages

We present TimeBraid, a series of unified time-series and language models that align pretrained language models and pretrained time-series foundation models through interleaved global residual attention layers. Each model inherits knowledge, instruction following, and reasoning from one side, continuous-signal perception and zero-shot forecasting from the other, and fuses the two in a shared representation space where both modalities are understood and generated. We study the design choices that make such unified modeling work: where to align the two representation spaces, how to ground language in temporal structure, how to balance understanding with generation, and how to keep joint optimization stable. The resulting recipe combines a unified prompting scheme for diverse time-series and text tasks, stabilized joint training, and supervision from 2.2M curated series--text pairs and 4.9M instruction-tuning samples. Across benchmarks spanning time-series perception, understanding, reasoning, and both context-aided and unimodal forecasting, TimeBraid remains competitive with far larger general-purpose models and task-specific counterparts.

0.27

Learning Options for Compositional Motor Control with Adapter Banks

cs.LG cs.ROq-bio.NC ·2026-09-15 ·Sreejan Kumar, Marcelo Mattar, Lea Duncker
hot direction similarity 0.60
Abstract

Learning flexible motor primitives is a hallmark of skilled motor control. Recent neuroscience theory proposes that motor primitives may be implemented as low-rank perturbations of a shared recurrent network, but leaves open how such a system is learned. We translate this principle into a novel architecture for learning motor skills end-to-end: a shared recurrent core modulated by a bank of residual adapters, each selected by a discrete latent code. Trained on closed-loop biomechanical control, the adapters develop emergent low-rank perturbations of the recurrent dynamics despite no architectural rank constraint, placing task representations in disparate subspaces of the shared core network. A simple high-level policy over the learned options, optimized while the whole network is frozen, sequences the low-rank adapters to produce novel out-of-distribution movements. We demonstrate the ability to generalize to novel motor sequences within the closed-loop control setting, improving on the generalization error of a task-input-conditioned multitask baseline by upto order of magnitude.

0.31