CS Paper Radar

Coordinated Lane-Level Variable Speed Limits and Ramp Metering for Successive Weaving Segments Considering Merging/Diverging Risks: A Hybrid Model Predictive Control and Multi-Agent Reinforcement Learning Approach

Guodong Ma, Baofeng Sun, Wenyu Yang, Zhihong Yao

cs.LG math.OC ·submitted 2026-09-28 13:32 UTC · v1

Abstract

Successive weaving segments (SWSs) on urban expressways are bottlenecks prone to recurrent congestion and collisions, requiring fine-grained active traffic management (ATM). Existing approaches struggle to balance the adaptive performance of data-driven optimization with the resilience and transferability of model-based control. We propose a hybrid framework to coordinate lane-level variable speed limits (VSLs) and ramp metering across SWSs. First, we reconstruct L-METANET, a lane-level macroscopic traffic flow model that captures free and forced lane changes. Second, we combine XGBoost-SHAP with a random-parameters binary logit (RPBL) model to derive analytical equations for merging and diverging collision risks and formulate system cost and reward functions. Third, we develop MPC-STMAPPO, a hierarchical controller integrating model predictive control (MPC) and multi-agent reinforcement learning (MARL). Its upper MPC layer uses L-METANET for long-horizon rolling optimization and generates baseline commands; its lower spatiotemporal MAPPO (ST-MAPPO) layer, enhanced with Mamba cells and graph attention, produces residual actions for short-horizon adjustment. Real-world experiments on the 18-km Eastern Expressway in Changchun, China, show that L-METANET accurately reproduces lane-changing-induced flow redistribution and capacity drops, with state evolution aligned with ground truth. XGBoost-SHAP-RPBL achieves AUCs above 0.80 in most tasks, outperforming conventional logit models. MPC-STMAPPO converges faster and performs better across multiple metrics than MPC- and MARL-based baselines. Under randomly fluctuating demand, it also significantly outperforms pure MARL in generalization, demonstrating strong potential for industrial deployment.

Closest papers

Nearest neighbours by meaning across the whole index.

Unified Optimization of Traffic Signal and Lane Channelization for an Isolated Intersection Using Reinforcement Learning

cs.LG ·2026-11-10 ·Jiwon Kang, Jiyoung Kim, Hyunseok Jun, Keemin Sohn
similarity 0.77
Abstract

Optimizing traffic signal control has been a long-lasting critical issue in traffic engineering, and a number of robust methodologies have emerged to solve the problem so far. On the other hand, the road design for lane channelization is a static problem that is difficult to solve jointly with the real-time traffic signal control. If traffic volume does not vary, it is possible that both problems can be solved in an integrated manner as a single static optimization problem. Recently, researchers have devised a mixed integer linear problem (MILP) to jointly determine the traffic signal and road width designs. However, this joint optimization problem must be solved repeatedly whenever the traffic flow pattern varies. Solving the MILP may also take impractical computing time as the problem size gets larger. We present a robust methodology for solving the joint optimization problem using reinforcement learning (RL) on a real-time basis. Abrupt changes in lane markings with an incompatible signal phase can jeopardize drivers. To sidestep such a dangerous action change, we train a model with human feedback to filter infeasible action transitions in a rule-based manner. We devised a deep Q-network that can accommodate joint actions for traffic signal and lane marking plans. To ensure the implementation of the proposed operation, a variable sign for lane markings is necessary at each stop line of an intersection. The experimental results confirm that the proposed methodology enhances the intersection performance by increasing throughput.

0.19

Learned Terminal Costs for Short-Horizon Model Predictive Control of Autonomous Trucks in Interactive Highway Traffic

cs.LG cs.AIcs.ROeess.SY ·2026-10-29 ·Hooman Hamze, Benjamin Houshmand, Deepthi Pathare, Erik Börve, Morteza Haghir Chehreghani +1 more
similarity 0.76
Abstract · 13 pages, 2 figures, 8 tables

Autonomous highway driving for heavy-duty trucks is a challenging control problem. The greater mass, dimensions and intricate dynamics increase the requirements for cooperation from surrounding traffic in critical maneuvers, requiring a careful balance between safety constraints and efficient long-horizon decision making. Model Predictive Control (MPC) provides rigorous constraint satisfaction but relies on terminal cost which can limit its effectiveness at short, computationally affordable horizons. Reinforcement Learning (RL) captures long-term driving behavior but offers no constraint guarantees and tends toward overly conservative policies when trained under sparse collision penalties. In this paper, we propose a hybrid MPC and RL framework for highway driving in autonomous trucks, where a finite-horizon MPC, solved via dynamic programming, uses a pre-trained RL model for terminal cost estimation. We evaluate the framework in a simulation environment with reactive surrounding traffic and compare it against pure MPC with zero terminal cost, rollout with terminal cost estimation using a base policy, and pure RL. Results show that the hybrid approach achieves zero collisions with the lowest cost of any configuration.

0.12

Reinforcement Learning-Based Control of CAV Platoon Joining Maneuvers in Mixed Traffic

cs.LG cs.AImath.OC ·2026-08-27 ·Biao Yin, Abderrahmane Kasmi, Nadir Farhi
similarity 0.73
Abstract

Connected and automated vehicle (CAV) platooning offers a promising approach to improving road safety and traffic capacity. However, platoon control in real-world traffic is challenging due to uncertainty and heterogeneous driving behaviors. Reinforcement learning (RL) has strong potential for addressing such control problems, but its practical deployment raises challenges related to safety and learning efficiency. This paper proposes a generic modeling and simulation framework for investigating CAV platoon joining maneuvers and comparing deep reinforcement learning (DRL)-based control algorithms. The problem is particularly challenging in mixed-traffic environments, where CAVs coexist with human-driven vehicles exhibiting heterogeneous longitudinal and lateral behaviors. The objective is to achieve safe and efficient joining maneuvers by either incorporating penalties for risky behaviors into the learning process or using an external safety controller to constrain the learned policy. An agent-based modeling framework coupled with the Simulation of Urban MObility (SUMO) simulator is used to evaluate Deep Q-Network (DQN), Double Deep Q-Network (DDQN), and Proximal Policy Optimization (PPO). Results show that PPO outperforms DQN and DDQN, achieving a joining success rate of approximately 98 % and a collision rate below 1 %, largely due to risk-related penalties incorporated into the reward function. However, this improved performance requires more decision steps to complete the maneuver, revealing a trade-off between safety, joining effectiveness, and decision efficiency. An external safety controller effectively prevents collisions, although its interventions may reduce joining efficiency. The results highlight the importance of jointly considering safety and efficiency when designing RL-based controllers for CAV platoon joining in mixed traffic.

0.06

Prescribed Performance Guided Safe Reinforcement Learning for Roundabout Navigation of Autonomous Vehicles

cs.RO ·2026-10-20 ·Wenhan Li, Yangrui Zhang, Hang Zhou, Peiran Liu, Yiding Ji
similarity 0.71
Abstract · 8 pages, 10 figures

Connected and Autonomous Vehicles (CAVs) face significant challenges in roundabout navigation due to dense traffic interactions and unpredictable agent behaviors, necessitating high levels of adaptability and safety. While the combination of Reinforcement Learning (RL) and Model Predictive Control (MPC) offers a promising approach, existing frameworks suffer from a structural mismatch where pointwise RL decisions fail to align with MPC's trajectory level optimization, and often lack rigorous safety guarantees. This work proposes RLTrajGen-PPMPC-CBF, a hierarchical framework for safe, smooth and comfort roundabout navigation, which regulates the coupling between RL-based planning and MPC-based control. We employ Prescribed Performance Constraints (PPC) to enforce a gradual, stable convergence between reference trajectories and control outputs, ensuring MPC tracking performance remains consistent with intended behavior from RL. Additionally, a Control Barrier Function (CBF) layer provides real-time safety corrections that are fed back into the RL process. Simulation results in the MetaDrive environment demonstrate that our method achieves a 6.25\% increase in average success rate and a 65.87\% reduction in Motion Sickness Dose Value (MSDV) compared to the optimal baseline, meanwhile accelerating training convergence.

0.23

SIGMA: Symmetry-aware, Intelligent, Geometric, Multi-faced Adaptive Traffic Control for Reliable Intelligent Transportation Systems

cs.LG stat.ML ·2026-08-18 ·Pratham Payra, Jagadish, Tanmay Sen, Tanujit Chakraborty
similarity 0.69
Abstract

Traffic signal control is a challenging sequential decision-making problem requiring a balance between throughput, vehicle delay fairness, signal stability, predictable switching, and emergency vehicle prioritisation. Existing reinforcement learning (RL) approaches often rely on predefined objectives and offer limited adaptability to changing operational priorities. They may also generalise poorly across geometrically equivalent intersections. To address these limitations, SIGMA (Symmetry-aware, Intelligent, Geometric, Multi-faced Adaptive traffic control) is proposed as a multi-faced RL framework augmented with a large language model (LLM) for dynamic objective adaptation and orientation-invariant policy learning. Natural language emergency instructions are translated into priority vectors that guide a multi-faced actor-critic controller. Symmetry-aware rotational augmentation supports transfer across homogeneous four-way intersections, while an offline-to-online learning strategy enables stable initialisation and adaptation to evolving traffic conditions. Structural properties concerning emergency service levels, graceful degradation under LLM failure, and sensitivity to traffic demand perturbations are established, alongside bootstrap-based statistical validation. Experiments in SUMO across four urban intersections modelled on Kolkata traffic scenarios compare SIGMA with fixed-time, actuated, and Deep Q-Network controllers. Reductions in average waiting times, emergency waiting times, and queue lengths are observed, alongside improved traffic throughput. Empirical reliability and ablation analyses further assess robustness to component failures and geometric orientation. SIGMA integrates language-guided objective adaptation, multi-faced policy learning, and statistical reliability assessment into an adaptive framework for intelligent traffic control.

0.16

Safeguarding Multi-Agent Reinforcement Learning for Connected and Automated Vehicles with Recursively Feasible Control Barrier Functions

cs.RO ·2026-11-03 ·Jianye Xu, Yousef Sayed, Bassam Alrifaee
similarity 0.68
Abstract

Multi-Agent Reinforcement Learning (MARL) shows promise in coordinating connected and automated vehicles but offers no safety guarantee. Although Control Barrier Functions (CBFs) can add this guarantee, their optimization problem can become infeasible in practice due to control bounds. Moreover, they often overapproximate vehicle and road geometry, which makes them conservative. This paper proposes the Backup-Horizon CBF (BH-CBF), which evaluates safety along a horizon until all vehicles stop. BH-CBF is recursively feasible under control bounds, and it considers the exact vehicle and road geometry. Based on BH-CBF, we derive a fully decentralized safety filter that only needs a single broadcast of vehicle states and routes per step and stays real-time in dense traffic. We use BH-CBF to safeguard a MARL policy over its life cycle: as a reward during pre-training, as a run-time filter at deployment, and as a filter in the training loop that keeps fine-tuning on unseen maps collision-free. In simulations on real-world maps of a roundabout and an intersection, BH-CBF avoids all collisions of unsafe MARL policies and keeps its safety guarantee at every step. Existing filters either collide, lose the guarantee, or fall back to full braking far more often. In recorded roundabout traffic with 1264 human-driven vehicles over 115 km, our safeguarded policies shorten the median travel time by 14% compared with the human drivers, without any collision.

0.25

Composite-Gradient Learning for Shared Control Authority Between Deep Reinforcement Learning and Model Predictive Control

cs.LG eess.SY ·2026-09-15 ·Giray Önür, Azita Dabiri, Bart De Schutter
similarity 0.68
Abstract

Integrated deep reinforcement learning (DRL) and model predictive control (MPC) methods are increasingly used to control autonomous systems by combining their complementary capabilities. DRL learns control policies through interaction with the environment. MPC uses a system model to optimize control inputs while accounting for constraints. In DRL-MPC frameworks with shared control authority, both the DRL agent and the MPC controller each determine part of the control inputs. However, common learning formulations treat MPC as part of the environment and therefore do not explicitly account for MPC's contribution to control or its interaction with the DRL agent. This paper proposes a novel composite-gradient learning (CGL) method that integrates the MPC controller into the learning process by representing the DRL and MPC control inputs as a joint action and accounting for their interaction when updating the DRL agent during training. CGL is evaluated on two multi-class freeway traffic networks with different strengths of interaction between the DRL and MPC control inputs and it is compared with alternative methods that treat MPC as part of the environment or that only partially incorporate MPC into learning. The results show that CGL offers limited benefit under weak interaction, but learns higher-performing control policies than the alternative methods in a subset of training runs under strong interaction, although the average control-performance gains remain modest.

0.06

Knowledge-Data-Dual-Driven Reinforcement Learning for Autonomous Vehicle Control in Mixed Traffic

cs.RO eess.SY ·2026-08-14 ·Jie Fang, Wei Zheng, Mengyun Xu, Eui-Jin Kim
similarity 0.68
Abstract · 16 pages, 17 figures

In mixed traffic, decision-making for autonomous vehicles (AVs) confronts three interrelated challenges. First, physics-based priors incorporated into reinforcement learning (RL) models fail to capture latent interactive vehicle intentions and diverse driver behaviors, limiting the proactive reasoning capabilities. Second, abrupt maneuvers by surrounding vehicles cause non-stationarity, leaving long-tail safety events under-explored. Third, hybrid action spaces destabilize unified RL training due to the different temporal scales of continuous car-following and discrete lane-changing maneuvers. To address these issues, we propose Knowledge-Data Dual-driven Reinforcement Learning (KDDRL). First, a conditional deep generative model synthesizes intention-aware future trajectories, converting passive perception into proactive predictive states. Second, a knowledge-data dual-driven paradigm operates on these predictive states, fusing probabilistic data-driven insights with physical constraints to guide safe exploration through safety-critical scenarios. Third, a coupling module compresses both intention-aware trajectories and physical constraints into compact shared embeddings. This unified representation enables asynchronous multi-timescale optimization of continuous car-following and discrete lane-changing while preserving mutual information. Evaluations on dataset-calibrated simulations demonstrate that KDDRL effectively handles intention uncertainty, accelerates training convergence, and outperforms conventional baseline methods in terms of safety, efficiency, and comfort.

0.10