CS Paper Radar

Neurons Are All You Need

Joseph B. Alzagatiti, Alain C. Glanzman, Perry A. Demsko, Pablo I. Salanova, David L. Glanzman

cs.AI ·submitted 2026-10-30 16:35 UTC · v1

Abstract

For decades, artificial neural networks have operated under the assumption that durable, learned capability resides exclusively in adaptive synaptic parameters, primarily due to the established neurobiology of synaptic connectivity. This design forces models to remain stateless during deployment, incurring well-known limitations such as catastrophic forgetting and expensive retraining. However, recent neurobiological research by Glanzman and colleagues, and others, shows that memories can be transferred between individual organisms in a synaptically independent manner. Inspired by this research, we sought to demonstrate memory transfer of persistent memory between AI model instances, leveraging a neurocentric version of memory. In this paper, we propose NURESA (Neural Unit with Retentive Self-Adaptive Architecture), a novel neural layer that eschews sole reliance on synaptic weight updating by introducing persistent intra-unit memory as a first-class, computational primitive. By structurally separating computation from persistent memory, NURESA enables autonomous memory accumulation and consolidation into transferable stores of memory registers. Experiments across MLP, GPT-2, and modern frontier models like Qwen2.5 demonstrate qualitatively new capability: zero-shot transfer of learned capability into a naive model instance without retraining or access to the original training data; this transfer is enabled by a sparse set of parameters in NURESA's memory register that the model selects on its own based on which neurons were most active. The memory transfer capability of NURESA also protects against domain loss when transformer models undergo learning via fine-tuning. Finally, the memory transfer process completes in approximately one minute even for transformer models with millions to billions of parameters, producing a transfer file orders of magnitude smaller than the original model.

Closest papers

Nearest neighbours by meaning across the whole index.

In Two Minds about Lifelong Learning: Exploring Hemispheric Redundancy and Specialisation in Neural Models

cs.LG cs.AI ·2026-08-20 ·Benjamin Smith, Levin Kuhlmann, Kaushik Roy, Gideon Kowadlo
hot direction similarity 0.71
Abstract · 21 pages before references/appendix, 9 figures

Persistent intelligent systems require the ability to learn continually, but current machine learning approaches face significant challenges in this area compared to biological learning systems. Machine learning algorithms typically trade off retention of previously learned information and adaptation to new or changing data patterns. When continual learning capabilities are absent, algorithms must undergo retraining using the entire data set, an approach that becomes impractical when original training data are unavailable due to storage constraints, financial or computational costs, or privacy restrictions. However, biological animals can learn continually, without experiencing catastrophic forgetting. This paper attempts to build a high-level framework for how animals learn and preserve knowledge by modelling neural components and states that are known to be related to memory consolidation. We focus on three concepts: experience replay, REM sleep, and bilaterality. We propose 4MAS (4 Module Awake/Sleep), a novel macroarchitecture demonstrating how machine learning models might benefit from asymmetric hemispheres, each with their own long- and short-term memory mechanisms, and how a period of sleep between incremental learning tasks might benefit memory consolidation. Finally, we present results showing that our architecture achieves statistical improvements over established generative replay baselines on Split-MNIST (98.3%) and Split-Fashion-MNIST (84.9%), while reducing forgetting by more than half relative to the strongest replay baseline and demonstrating monotonic capacity scaling on Split-CIFAR-100 (29.29%).

0.31

Engram Neuron Inspired Adaptive Memorization for Lifelong Learning

cs.CV ·2026-10-10 ·Suiyang Jiang, Bomin Wang, Hangqi Zhou, Jianfeng Feng, Xiahai Zhuang
similarity 0.70
Abstract

Lifelong learning enables artificial systems to adapt to changing environments. To achieve this, an effective artificial learner should dynamically reorganize past and new experiences, maintaining stability while enabling flexible knowledge integration. However, existing efforts primarily mitigate catastrophic forgetting, lacking the ability to adaptively update memory akin to biological systems. Here, by integrating the full memory life cycle of engram neurons into artificial learners, we propose a framework that constructs a structured memory from gradient representations, which regulates parameter updates and guides memory management. Through comprehensive evaluations on four benchmarks, our approach shows promising lifelong learning capability, maintaining stability over long task sequences and supporting flexible adaptation. Beyond the performance gains, our approach offers a computational perspective on robust and flexible lifelong learning, echoing mechanisms observed in biological memory systems.

0.18

Hippocampal consolidation without replay in fully fine-tuned networks

cs.AI ·2026-11-01 ·Keuntae Kim, Yong Suk Choi
code/data similarity 0.66
Abstract · 37 pages, 5 figures, 1 table; Extended Data and Supplementary Information included. Code: https://github.com/kkt94/deltahippo

Networks that keep learning overwrite what they know. Prevailing remedies replay stored data, distil from old models or freeze the backbone. The brain suggests a different division of labour: the hippocampus indexes where experience lies in cortex while the cortex keeps learning. We show that this division lets a fully trainable network keep learning without cross-task replay, a teacher or inference-time memory. After each task, a training-only hippocampal memory indexes the input geometry of every group of synapses sharing an input, residual energy included, and limits each synapse's movement per direction before and after the optimizer. This class-free geometry carries most of the protection; with conflict detection on top, the method cuts absorption of established classes by near-duplicates several-fold against every compared method. To our knowledge, this is the first demonstration of one principle keeping whole networks learning across four architecture families and models up to 8 billion parameters. Methods that freeze the backbone hold up only as far as its representation already separates the classes; here the whole network keeps learning to tell near-duplicates apart, leading the frozen-backbone baseline by 17.5 points on TACRED. Without the memory, much old knowledge stays decodable but beyond the network's reach.

0.27

Dual-Layer Agentic Memory with Fast Write Routing and Slow Consolidation

cs.CL ·2026-08-23 ·Wenzhi Li, Dong Nie, Rui Lan, Tongtong Lyu, Peiyao Wang +5 more
position similarity 0.66
Abstract

Large language model (LLM) agents operate in dynamic environments where knowledge continuously evolves. Existing memory systems typically treat external memory as a monotonically growing repository, inevitably leading to retrieval degradation and increasing computational costs over time. We argue that the core challenge is not retrieval alone, but managing the knowledge lifecycle: deciding what to externalize, update, or ultimately internalize. Inspired by Complementary Learning Systems (CLS) theory in neuroscience, we propose Dual-Layer Agentic Memory, a framework that shifts memory management to the write phase through cost-aware epistemic routing and periodic parametric consolidation. Incoming information is categorized as non-write, write-new, or write-update, and routed through a small-to-large model cascade that minimizes routing overhead while filtering redundant memories. A subsequent write-back phase selectively consolidates high-value external memories into model parameters via supervised fine-tuning. Experiments demonstrate the dual efficiency of our approach: a 1.7B/8B cascade prunes up to 68% of redundant external memory while escalating fewer than 50% of inputs, yet retains over 98% of the downstream QA Exact Match (EM) achieved by an exhaustive retention baseline. We further show that periodic consolidation successfully internalizes external knowledge, allowing the router to adaptively suppress redundant writes as the model's epistemic boundaries evolve. Overall, our framework presents a unified paradigm for agent memory: selective externalization followed by selective internalization. Code and dataset will be released upon acceptance.

0.46

Memory-Coupled Unlearning for Data-Absent Identity Forgetting

cs.CR ·2026-10-09 ·Chun-I Fan, Lu-En Chen, Ming-Feng Tsai, Tomohiro Morikawa
similarity 0.65
Abstract

Machine unlearning aims to reduce the influence of designated training data from a trained model while preserving its utility on retained data. Identity-level unlearning is particularly challenging when only limited target-specific information is available when a forgetting request is processed. In this work, we propose Memory-Coupled Unlearning (MCU), a framework that augments a host model with a Modern Hopfield Network (MHN) serving as a controllable external identity-prototype memory. During prototype construction, feature representations associated with each training identity are aggregated into identity prototypes and stored in the MHN. A Prototype Adaptation Network (PAN) transforms the retrieved memory representation, while a Dependency Gating Unit (DGU) controls its contribution to downstream prediction. When an identity is requested to be forgotten, MCU first removes the corresponding prototype from the external memory and subsequently readapts the model using limited support information associated with the target identities together with retained data. Importantly, in this work, ``data-absent'' specifically refers to the absence of the complete original forget set during request-time gradient-based readaptation; retained data may still be used to preserve downstream-task utility. Experiments based on the identity-partitioning and one-shot support construction of 1-SHUI show that MCU achieves a favorable empirical trade-off among model utility, membership distinguishability, and request-time computational efficiency compared with the evaluated baselines. These results provide empirical evidence for the potential of controllable external memory in machine unlearning, rather than a formal guarantee of complete information removal.

0.26

Continuous Memory Machines

cs.AI ·2026-10-06 ·Ciaran Regan, Kai Arulkumaran, Luke Darlow, Stefania Druga, Sebastian Risi +1 more
publishedcode/data similarity 0.65
Abstract · NeurIPS 2026 Workshop: Personalized, Aligned, Long-Term Memory for AI Systems (PALM)

Recurrent neural networks typically compress information into a single vector-valued recurrent state, forcing short-term computation and long-term retention to share the same representation. Past extensions alleviate this bottleneck by increasing the memory capacity or separating timescales, but lack the combination of rapid neuron-level processing and longer-term retention found in biology. To that end, we introduce the Continuous Memory Machine (CMM), a recurrent architecture with matrix-valued short- and long-term memory states serving distinct functional roles. Building on the Continuous Thought Machine (CTM), the CMM's short-term memory tracks recent neural activity, with uniquely parameterized neuron-level models learning to use these activity patterns for computation. A persistent long-term memory stores information for later use, with a Transformer jointly updating both memory stores, providing an expressive bidirectional read--write mechanism such that each store can reorganize its own contents and both read from and write to the other. Across algorithmic, in-context learning, and recurrent reasoning tasks, the CMM outperforms a broad suite of baselines, exhibiting stronger generalization than prior memory-augmented networks while preserving the CTM's interpretable attention patterns. Code is available at https://github.com/SakanaAI/continuous-memory-machines.

0.43

Selective Knowledge Control for Continual GUI Agent Learning over Application Streams

cs.CV cs.AI ·2026-09-06 ·Zirui Shang, Xin Shu, Yang Liu, Zhi Gao, Xinxiao Wu +1 more
benchmark similarity 0.64
Abstract

Continual learning is a crucial capability for Graphical User Interface (GUI) agents to adapt to evolving applications while retaining knowledge acquired from previous applications. Such application streams pose a challenging knowledge modeling problem: new applications often share underlying knowledge with past ones, yet also introduce specific knowledge that must not interfere with historical knowledge. In this paper, we propose activation-conditioned selective knowledge control, a lightweight method that achieves selective knowledge retention via neuron-level gradient manipulation. Our method maintains a compact historical knowledge state to protect highly activated MLP neurons that preserve previous knowledge. When a new application arrives, it performs real-time gradient surgery conditioned on forward activation. Concretely, the protected neurons are categorized into two types: unactivated neurons holding specific knowledge, whose gradients are truncated to prevent interference; and activated neurons holding shared knowledge, whose gradients are orthogonally projected to preserve stability while enabling adaptation. After each application stage, newly identified critical neurons are merged into the historical state for future learning. Empirical evaluations on multi-app sequential benchmark demonstrate that our method effectively mitigates catastrophic forgetting on prior applications while sustaining robust adaptation to new ones.

0.14

Position Paper: Neurotransmitters as a Missing Dimension in Artificial Neural Networks

cs.NE ·2026-09-17 ·Yupei Li, Manuel Milling, Berrak Sisman, Björn Schuller
publishedpositionhot direction similarity 0.64
Abstract · Accepted at IJCNN 2026

Artificial neural networks (ANNs), as core components of modern deep learning (DL) systems, lack the adaptive flexibility and long-term stability exhibited by biological systems. This limitation largely stems from the fact that conventional ANNs rely on uniform, local, and gradient-based parameter updates, while neglecting internal learning principles that are biological mechanisms such as neurotransmitters signalling or neuroplasticity. Consequently, many existing approaches focus on architectural expansion or mathematical fine-tuning techniques such as regularisation or parameter isolation. Inspired by the superior adaptability and plasticity of mammalian brains, we posit that neuromodulation with neurotransmitters constitutes a third axis of learning, complementary to neural activity and synaptic plasticity, and should be explicitly modelled in artificial neural networks. In this positional paper, we argue that incorporating neuromodulatory principles into ANN design represents a promising and underexplored research direction, and we advocate for greater attention to this perspective in the development of adaptive and continual learning systems.

0.44