CS Paper Radar

Planning as Dynamics Relaxation: Hippocampal Recurrent Network Realizes Optimal Goal-Directed Navigation

Yuhang He, Junfeng Zuo, Tianhao Chu, Si Wu

cs.AI q-bio.NC ·submitted 2026-08-30 16:48 UTC · v1
theory

Abstract

Neural correlates of spatial cognitive map are well documented, yet exactly how neural circuits perform spatial navigation in complex environments - e.g., reaching a goal while avoiding obstacles - remains largely unclear. Here, we show that a hippocampal network with appropriate recurrent connections can naturally achieve optimal goal-directed navigation via its relaxation dynamics. Specifically, we consider that the recurrent weights between the neurons represent the transition probabilities between spatial locations encoded by neurons; obstacles such as walls and blocked corridors are therefore reflected by the vanishing of connection weights. This connection pattern can be learned in the hippocampus via behavioral-timescale synaptic plasticity (BTSP) while the animal is exploring the environment. When a goal signal is presented, the network dynamics will relax into an activity field representing the goal location. We prove that this field is mathematically equivalent to the desirability field of a Linearly-solvable Markov Decision Process (LMDP), and the local log-gradient of the field indicates the navigation direction. Both theoretical analyses and simulations demonstrate that this recurrent network dynamics-mediated navigation is efficient and robust in environments with complex obstacle layouts. Moreover, only low-rank updates of the network's connection pattern are needed when the environment has local changes. We hope this study offers insight into a general circuit principle for planning in abstract rational maps in the brain beyond spatial navigation.

Closest papers

Nearest neighbours by meaning across the whole index.

Learning to Plan from Random Exploration

cs.LG cs.ROstat.ML ·2026-09-29 ·Deqian Kong, Guangyan Sun, Sheng Cheng, Sirui Xie, Bo Pang +4 more
similarity 0.63
Abstract

Random exploration reveals how an environment can be traversed before a goal is specified. Can this experience support long-range planning without policy-improvement training? Our random-walk analysis explains what temporal relations contain: short horizons reveal geodesic geometry in the diffusion limit, while longer horizons reveal connectivity between regions before mixing removes these distinctions. We learn these relations with a conditional energy-based model that estimates temporal log-density ratios through horizon-conditioned embeddings. The model is trained on observation pairs by noise-contrastive estimation, without action or reward labels. The planner queries these learned relations at different horizons as it moves toward the goal. At test time, a separate local dynamics model predicts candidate action outcomes, and the temporal model evaluates their progress toward the goal by selecting or aggregating estimated improvements across horizons. The agent executes one action and replans with both models fixed. Experiments demonstrate long-range maze planning from random exploration using states and images. Learned score fields, embedding probes, and planned routes exhibit properties of a multiscale cognitive map. We further demonstrate egocentric navigation from random exploration and manipulation planning from suboptimal data.

0.17

Remember Where You Walk: Learning Spatial Memory for End-to-End Legged Robot Exploration

cs.RO ·2026-10-27 ·Leon Magnus, Nico Bohlinger, Jan Peters, Oleg Arenz
similarity 0.62
Abstract

Humanoids and other legged robots operating in complex environments must remember information that leaves their field of view and only becomes relevant much later. Reinforcement learning policies address this using memory networks such as LSTMs that are trained end-to-end with the policy to memorize such task-relevant aspects. Yet, such architectures often fail to preserve sufficient information over long horizons. In this work, we propose a network architecture based on Hadamard memory that specifically targets robot navigation. Our Local Spatial Matrix Memory (LSMM) enables RL agents to store generic task-relevant information at specific, position-based memory cells, enabling flexible and spatially grounded memorization. We evaluate LSMM on the task of autonomous maze exploration, where it matches the maze coverage of strong recurrent and finite-history baselines while being the only architecture that remembers the maze layout, reducing the reconstruction error threefold.

0.21

Learning infinite context windows in recurrent architectures via spatial neural computing

cs.LG eess.SYq-bio.NC ·2026-10-07 ·Aleix Salvador-Pomarol, Arthur N. Montanari, Earl K. Miller, Adilson E. Motter, Jorge Cortés
similarity 0.62
Abstract · 12 pages, 4 figures

Recurrent neural networks (RNNs) offer linear-time scaling with sequence length while requiring only constant memory, yet they struggle to capture long-range dependencies due to vanishing gradients and limited receptive fields. To address these limitations, we introduce a second-order recurrent model in which the standard neuron-to-neuron communication is replaced by a spatially evolving field governed by (discretized) partial differential equations. Drawing inspiration from the role of cortical waves in brain computation, this mechanism allows structured spatiotemporal patterns to serve as an implicit, high-capacity memory. We show that the resulting model is equivalent to a structured infinite-order RNN in which the current state depends explicitly on its entire history of past states, yielding an effectively unbounded receptive field with a fixed number of parameters. We further derive constructive conditions to ensure marginal stability, constraining the gradient spectrum on the unit circle and thereby eliminating vanishing and exploding gradients. Empirically, the proposed architecture outperforms other recurrent models on long-horizon benchmarks while using substantially fewer parameters, demonstrating that spatial dynamics can effectively bridge the gap between efficient inference and long-term memory.

0.31

Hippocampal consolidation without replay in fully fine-tuned networks

cs.AI ·2026-11-01 ·Keuntae Kim, Yong Suk Choi
code/data similarity 0.62
Abstract · 37 pages, 5 figures, 1 table; Extended Data and Supplementary Information included. Code: https://github.com/kkt94/deltahippo

Networks that keep learning overwrite what they know. Prevailing remedies replay stored data, distil from old models or freeze the backbone. The brain suggests a different division of labour: the hippocampus indexes where experience lies in cortex while the cortex keeps learning. We show that this division lets a fully trainable network keep learning without cross-task replay, a teacher or inference-time memory. After each task, a training-only hippocampal memory indexes the input geometry of every group of synapses sharing an input, residual energy included, and limits each synapse's movement per direction before and after the optimizer. This class-free geometry carries most of the protection; with conflict detection on top, the method cuts absorption of established classes by near-duplicates several-fold against every compared method. To our knowledge, this is the first demonstration of one principle keeping whole networks learning across four architecture families and models up to 8 billion parameters. Methods that freeze the backbone hold up only as far as its representation already separates the classes; here the whole network keeps learning to tell near-duplicates apart, leading the frozen-backbone baseline by 17.5 points on TACRED. Without the memory, much old knowledge stays decodable but beyond the network's reach.

0.27

Free-Energy-Gated Plasticity for Real-Time Online Motor Learning in Physical Human-Robot Interaction

cs.RO cs.HC ·2026-08-24 ·Hiroki Sawada, Jun Tani
similarity 0.61
Abstract

Fully online embodied learning requires synaptic adaptation to acquire new behaviors while preserving previously learned dynamics during ongoing interaction. We extend the Predictive-Coding-inspired Variational Recurrent Neural Network (PV-RNN) to continuously adapt its synaptic weights and propose Free-Energy-Gated Plasticity (FEGP), which regulates the effective learning rate according to variational free energy. In real-time physical human-robot interaction, a randomly initialized network acquired three cyclic motor patterns without offline pretraining, replay, or task-boundary signals, with all three patterns emerging in autonomous rollouts. Controlled experiments over ten randomized teaching streams and five network initializations per stream showed that FEGP substantially improved repertoire coverage and retention of previously acquired patterns after they left the recent observation window. Neither a constant learning rate matched to the gate's time-averaged effective rate nor replay of the same gain values with disrupted temporal organization reproduced these improvements. These results indicate that the temporal allocation of plasticity relative to model-environment mismatch, rather than simply its average magnitude or distribution, is critical for maintaining previously acquired behaviors during continued online learning.

0.32

Do Better Goal Representations Improve Goal-Conditioned Reinforcement Learning?

cs.LG cs.AI ·2026-09-30 ·Syed Nazmus Sakib, Abdul Monaf Chowdhury, Nafiul Haque, Shifat E Arman, Md Mehedi Hasan
similarity 0.60
Abstract · 21 pages, 12 figures, 6 tables

Goal-conditioned reinforcement learning (GCRL) relies heavily on how target goals are represented to the policy. While recent methods encode goals via temporal distance, occupancy, or controllability, it remains unclear how much downstream performance actually depends on representation quality. We study this in offline GCRL by constructing an exact temporal-distance goal representation in deterministic mazes. We then systematically corrupt its geometric quality while keeping the downstream learner fixed. Across OGBench navigation tasks and two algorithms, large changes in goal-representation quality produce almost no change in performance. However, applying the same interventions to the agent's current state more than doubles success, revealing the state pathway as the true bottleneck. Building on this insight, we show that simple random Fourier positional encodings substantially improve performance on the hardest navigation tasks without map information or objective modifications. Overall, our findings suggest that in state-based offline navigation, improving how the agent's current state is represented matters far more than refining the goal representation. Code will be released soon.

0.35

Shaped by What's Missing: Topological Invariance Simplification Discovers Behavioral Transitions

cs.LG ·2026-10-31 ·Kaiwen Bian, Eric J. Leonardis, Yuanjia Yang, Charles Zhang, Yusu Wang +1 more
published similarity 0.60
Abstract · 37 pages, including references and appendix (9 pages main paper). Accepted to NeurReps 2026 (Symmetry and Geometry in Neural Representations…

Understanding how animals move between behaviors is a central question in neuroethology, yet most segmentation methods first partition time into discrete states, leaving transitions as the unmodeled residue between them. We take the opposite view: in a suitable feature space, sustained behaviors are dense regions of the trajectory's density landscape, and transitions are the sparse corridors and loops that connect them. Because this structure is a property of the landscape's shape rather than of any particular coordinates, it can be read off directly with topology and should persist across feature spaces. We build this landscape from frame-by-frame trajectories, extract its discrete-Morse graph, and keep two kinds of structure: paths, the corridors a trajectory follows between dense behavioral regions, and cycles, closed routes whose entry and exit take different ways. Clustering these structures yields transitions directly, with no upstream state partition. On a rat behavior dataset across three feature spaces, including the latent space of a neural-network controller that imitates the animal in a physics simulator, the approach recovers transitions at coverage comparable to state-space and topological-clustering baselines, while carrying substantially more information about the transition identity, forming clusters at least as tight as the baselines', and matching annotated transitions with less reliance on time-warping. More broadly, the results show that behavioral transitions are a topological feature of a representation: recoverable from the shape of its trajectories, and consistent across distinct feature spaces.

0.45

GLST: Global-Local Spatio-Temporal Memory Decomposition Framework for Online Chained-Goal Navigation

cs.RO cs.AI ·2026-10-13 ·Yousef Liang, Ryan Ardalan, Yarden Goraly, Chenhao Zhu, Yue Yin +4 more
published similarity 0.59
Abstract · Accepted at the Conference on Robot Learning (CoRL) 2026

Autonomous navigation in real-world environments often requires long-horizon operation under partial observability and evolving goals. Existing mapless navigation approaches predominantly assume fixed goals throughout an episode, limiting applicability to practical scenarios where goals change online during execution. We study online chained-goal navigation, where goals evolve through existing-goal redirection or new-goal emergence without explicit transition signals. Under this setting, we find navigation may require either continued exploration or strategic revisitation, creating conflicting memory requirements. We propose Global-Local Spatio-Temporal Belief Learning (GLST), a memory-decomposition framework that separates latent beliefs into global-local spatial and long-short temporal representations for long-horizon reasoning. We further introduce goal-conditioned rewards to balance exploration and exploitation under evolving goals. Extensive experiments in simulation across diverse terrains demonstrate improved chained-goal navigation performance over baselines. Real-robot experiments using the Spot quadruped further validate our memory decomposition framework.

0.30