CS Paper Radar

A Dynamic Fusion Large Language Model for Traffic Flow Prediction

Xue Qiu, Jianli Xiao

cs.LG ·submitted 2026-09-10 09:40 UTC · v1
published

Abstract

Traffic flow prediction is a core supporting technology for intelligent transportation systems. It uses historical data to infer future traffic dynamics in specific areas, thereby helping to alleviate congestion and improve resource allocation efficiency. Traditional neural networks struggle to break through accuracy limits due to their reliance on singular feature modeling, while large language models (LLMs) suffer from insufficient capture of spatial topological information and mining spatiotemporal correlation. This study proposes a Dynamic Fusion Large Language Model (DF-LLM) for traffic flow prediction. The model incorporates three core components: spatiotemporal embedding module, spatiotemporal fusion module, and LLM backbone. The spatiotemporal embedding module enables synergistic representation of multi-scale spatiotemporal features. The spatiotemporal fusion module integrates spatial topology and dynamic dependencies via graph convolution. The LLM backbone adopts a differentiated parameter adaptation strategy to balance training efficiency and traffic data adaptability. Additionally, it introduces a context aggregation attention module to strengthens global dependencies. More importantly, the LLM backbone takes the residual connections to mitigate the gradient vanishing in deep networks. Experiments show that DF-LLM has achieved better performance by comparing the metrics on all the four datasets.

Comment
Accepted by WISA 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Beyond Sunny Skies: Using Large Language Models to Predict Urban Traffic in Adverse Weather

cs.AI ·2026-11-09 ·Zhu Xiao, Kuan Feng, Tong Li, Chenxi Liu, Gang Wang +2 more
similarity 0.72
Abstract

Urban traffic flow prediction is a cornerstone of intelligent transportation systems, yet accurate forecasting under adverse weather conditions remains a significant challenge. Existing data-driven models, while effective for routine patterns, often struggle with the sparse data and complex non-linear shifts associated with rare meteorological events. These methods typically treat environmental contexts as shallow numerical features, failing to capture the underlying semantic reasoning-why and how specific adverse conditions alter mobility behaviors across regions with different functional compositions. To address this limitation, we propose Multi-STG, a unified framework that integrates Large Language Model (LLM)-derived semantic priors with graph-based spatiotemporal learning. Multi-STG introduces an LLM-empowered semantic enhancement module that transforms comprehensive context descriptions such as weather, POIs, and holidays into high-level semantic embeddings, effectively providing a knowledge-driven complement to data scarcity. Furthermore, we design a multimodal attentional fusion mechanism to dynamically weigh heterogeneous inputs and a semantic-enhanced prediction head with gated residual calibration to robustly adjust forecasts based on semantic guidance. Extensive experiments on real-world multi-city private car datasets demonstrate that Multi-STG outperforms state-of-the-art baselines, reducing MAE by 11.82\% on average relative to the strongest baseline across three cities, while maintaining stable performance under sparse and non-stationary conditions.

0.17

STHMoE: Hypergraph-Enhanced Heterogeneous Dependency Coordination for LLM-Based Urban Traffic Data Forecasting

cs.AI ·2026-09-14 ·Jiawen Chen, Qi Shao, Yongjian Chang, Mingtong Zhou, Duxin Chen +1 more
similarity 0.70
Abstract

Spatio-temporal traffic forecasting is a fundamental big data analytics task for intelligent transportation systems, where massive urban sensor streams exhibit heterogeneous, non-stationary, and structurally dynamic patterns. Although recent deep learning and large language model (LLM)-based methods have advanced traffic forecasting, they often remain temporally centered and lack effective coordination of temporal, spectral, pairwise spatial, and higher-order structural cues under evolving traffic regimes. To address this heterogeneous dependency coordination problem, we propose STHMoE, a Spatio-Temporal Hypergraph-Enhanced Mixture of Experts framework for urban traffic data forecasting. STHMoE decouples traffic dynamics into frequency-domain, time-domain, spatio-domain, and higher-order spatial representations, which are modeled by prompt-guided heterogeneous experts built upon a partially frozen LLM backbone. The first three experts leverage domain-specific statistical prompts, while the higher-order spatio expert uses a structural placeholder prompt and obtains dependency information from an adaptive hypergraph module. To capture evolving spatial structures in traffic data,, STHMoE jointly learns first-order graph dependencies and higher-order group interactions without predefined topologies. An entropy-aware MoE router with coefficient-of-variation load balancing adaptively fuses expert outputs while improving expert utilization and routing confidence. Experiments on 10 real-world traffic benchmarks show that STHMoE achieves competitive performance against temporal, spatio-temporal graph, and LLM-based baselines.

0.10

Learning Local Heterogeneity and Cross-Region Context for Large-Scale Traffic Forecasting

cs.LG cs.AI ·2026-09-23 ·Qi Feng, Zidong Wang, Bo Li, Xiaoguang Gao, Jiayu Zhang +2 more
benchmark similarity 0.70
Abstract

Traffic flow forecasting is essential to intelligent transportation systems. Large-scale traffic forecasting requires jointly modeling local spatial dependencies and cross-region context.Spatial dependencies between geographically neighboring nodes are heterogeneous due to differences in road identity and travel direction, while acquiring global information through allpairs node interactions incurs substantial computational costs. Therefore, capturing local heterogeneity while efficiently acquiring long-range context remains an important challenge in largescale traffic forecasting. To address these challenges, we propose LoReST, a Local-Region Spatial Temporal network that models spatial dependencies at two complementary granularities: node neighborhoods and road network regions. Specifically, relation-aware local aggregation captures heterogeneous dependencies within geographic neighborhoods through road and direction specific feature transformations. Cross-region interaction constructs region representations through mean pooling, exchanges long range context via inter-region attention, and broadcasts it back to nodes. By integrating local information aggregation with crossregion interaction, LoReST is able to effectively achieve spatial dependency learning in large-scale road networks. Experiments on four datasets of the LargeST benchmark show average relative reductions of 4.78%, 3.60%, and 5.75% in MAE, RMSE, and MAPE, respectively.

0.14

Rethinking Multi-View Fusion for Urban Representation Learning

cs.LG cs.AI ·2026-10-22 ·Sean Bin Yang, Zongyi Xu, Jilin Hu, Ying Sun, Kristian Torp +3 more
published similarity 0.69
Abstract · Accepted by PVLDB 2027

Urban region representation learning encodes heterogeneous urban signals, such as mobility flows, points of interest, and land-use patterns, into unified embeddings for downstream tasks, including check-in prediction, crime forecasting, and service demand estimation. Existing approaches rely on globally shared fusion and joint optimization, implicitly assuming uniform modality importance and naturally balanced multi-objective training. However, urban data is inherently heterogeneous. Our empirical analysis reveals two structural issues: (i) modality utility varies markedly across regions, causing static fusion to overemphasize signals from certain modalities; and (ii) heterogeneous objectives exhibit persistent gradient imbalance, causing representational bias and unstable training. We present PRISM, a self-adjusting framework for adaptive multi-view learning. PRISM adapts sparse Mixture-of-Experts (MoE) routing to learn region-specific view allocations, while a Proportional-Integral-Derivative (PID)-inspired regulation mechanism mitigates gradient conflicts during optimization. Experiments on three large-scale urban datasets show that PRISM achieves more competitive performance than strong multi-view baselines, while providing more stable optimization behavior and more balanced region representations.

0.16

Vision-Based Density Estimation for Adaptive Traffic Control

cs.CV ·2026-10-28 ·Mohammad Usaid, Deepak Silaych, Hrithik Mhatre, Archak Mittal
similarity 0.68
Abstract · 14 pages, 5 figures, 2 tables

Traffic congestion is becoming a critical challenge in rapidly growing urban centers, particularly in developing nations where rapid urbanization and inadequate transport infrastructure place increasing strain on transport networks. In highly heterogeneous and chaotic traffic flows, camera-based traffic-state estimation is limited by severe vehicle occlusion and overlap. In this paper, we propose a framework to optimize traffic flow directly from stochastic visual observations. The architecture is structurally decoupled into three primary components: (a) vehicle density estimation, (b) predictive dynamics modeling, and (c) policy optimization. The proposed methodology introduces a convolutional neural network (CNN) that learns the mapping from camera feeds to spatial density maps, utilizing an adaptive k-nearest-neighbor (k-NN) Gaussian kernel to handle varying congestion levels. This feeds into an ST3DNet predictive module for spatiotemporal forecasting, followed by a hierarchical Graph-Transformer Multi-Agent Reinforcement Learning (MARL) algorithm to optimize signal phases. The density estimation stage was evaluated on 40 annotated intersection images from Rambagh 4, Jaipur, India, with a reported five-fold validation mean absolute error (MAE) of $3.32 \pm 0.95$ vehicles and a mean absolute percentage error (MAPE) of 14.5%. These results pertain to the perception component; empirical training and evaluation of the forecasting and control stages remain future work.

0.31

A Multi-Resolution Multi-Domain Pre-Training Framework for Universal Traffic Forecasting

cs.LG ·2026-09-12 ·Zhouyang Liu, Jindong Han, Hao Wang, Xinyue Liu, Hui Gao +2 more
top venue similarity 0.68
Abstract · Accepted by ICDM 2026

Spatio-temporal traffic data are central to intelligent transportation systems, yet their heterogeneity poses significant challenges for large-scale modeling. Existing pre-trained models often rely on a homogeneous modeling paradigm to handle highly heterogeneous traffic data. This fundamental mismatch not only limits model generalization but also leads to computationally expensive and parameter-inefficient designs. To this end, we propose FlexST, a novel pre-training framework that introduces modularity and adaptivity for traffic modeling. Specifically, we first propose a multi-resolution spatio-temporal diffusion module that captures both short-term fluctuations and long-range trends, effectively reconciling inputs with divergent temporal and spatial resolutions. After that, we construct a domain-adaptive mixture-of-experts that dynamically routes data to specialized sub-networks, enabling selective knowledge transfer while preventing negative interference across diverse domains. Moreover, we devise a unified periodic encoding strategy that injects resolution- and domain-aware inductive biases to harmonize periodic inconsistencies across datasets. Extensive experiments on 23 real-world traffic datasets demonstrate that FlexST significantly outperforms state-of-the-art baselines in zero- and few-shot settings, showcasing superior generalization, adaptability and efficiency. This work offers a new direction for building general-purpose pre-trained models capable of handling the complexity and variability of urban traffic systems.

0.23

Synergistic Fusion of Topological Structure and Temporal Semantics of Mobility for Urban Region Embedding

cs.LG cs.AI ·2026-09-08 ·Namwoo Kim, Jeeyun Chang, Kanghoon Lee, Yoonjin Yoon
similarity 0.68
Abstract · Submitted to IEEE Transactions on Knowledge and Data Engineering

Urban region embeddings have shown promising results in diverse urban sensing tasks such as crime, income, and service-call prediction. Recent methods improve representation quality by integrating mobility data with auxiliary modalities, using cross-view attention or contrastive objectives to align heterogeneous features into a unified region representation. However, leveraging the temporal dynamics of human mobility remains under-explored. Regional inflow and outflow fluctuate throughout the day, and inter-region connections emerge, persist, and dissolve over time. Moreover, prevailing fusion strategies combine views additively and miss the joint signal that emerges only when views co-occur. To address these gaps, we propose Mobility Stream-Structure Synergy (MoSS), which derives complementary views from mobility data: a Sequence view that preserves each region's hourly inflow/outflow profile, and a Structure view based on zigzag persistence diagrams that capture how regional connectivity emerges, persists, and dissolves over time. A synergy module then extracts emergent representations from the co-occurrence of these views through multi-degree interactions, explicitly capturing higher-order signal across views. Extensive experiments on New York City and Chicago show that MoSS achieves state-of-the-art performance across three downstream tasks using mobility data alone, outperforming baselines that rely on auxiliary modalities.

0.10

Structure-Guided Spatiotemporal Attention Graph Neural Network for Traffic Flow Prediction

cs.LG cs.AI ·2026-08-14 ·Xuanmian He, Can Li, Wanjing Ma
similarity 0.66
Abstract

Deep spatiotemporal models integrating graph convolutions and attention mechanisms have demonstrated excellent performance in network-level traffic flow prediction, owing to their exceptional ability to capture complex spatiotemporal dependencies. Despite their predictive success, deployment of such models in safety-critical urban systems remains constrained by their inherent lack of transparency. Existing post-hoc diagnostic methods often struggle with spurious correlations and fail to unveil the intrinsic decision-making mechanisms governing traffic dynamics, resulting in suboptimal interpretability and limited operational trustworthiness. To address these challenges, this paper proposes the Structure-Guided Spatiotemporal Attention Graph Neural Network (SGSAN). Departing from traditional architectures that rely on unconstrained adaptive graphs, SGSAN explicitly learns a static Directed Dependency Graph (DDG) to identify the invariant macroscopic propagation paths of traffic states. We further introduce an InfoNCE-based soft-coupling mechanism that anchors the model's dynamic spatiotemporal attention to this structural prior, offering a mechanistic account of the model's decision-making process while ensuring robust forecasting by aligning attention-based reasoning with identified macroscopic dependencies and preventing over-reliance on ephemeral local noise. Furthermore, a decoupled two-stage optimization framework is developed to resolve the fundamental conflict between structural discovery and predictive error minimization. Extensive experiments on multiple real-world datasets demonstrate that SGSAN achieves state-of-the-art predictive accuracy while providing built-in interpretability that organically aligns with the physical logic of traffic networks.

0.04