CS Paper Radar

Second-order optimization of variable projection SVM models and road abnormality detection

Andrea Angino, Matthias Voigt, Rolf Krause, Tamás Dózsa

eess.SP cs.LG ·submitted 2026-10-07 07:57 UTC · v1
publishedunusual

Abstract

We introduce a novel second-order optimization framework for minimizing so-called variable projection functionals. We demonstrate that the proposed framework is especially usefulfor the training of variable projection based kernel methods. In particular, the problem of efficiently training variable projection support vector machines (VP-SVMs) is considered. We show the effectiveness of the proposed training methodology in a real-world application, namely we demonstrate how second-order trust region algorithms can be used to train VPSVM models to recognize road surface abnormalities based on 1D signals obtained from a tire sensor.

Journal ref
Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2026), Barcelona, Spain, 2026
DOI
10.1109/ICASSP55912.2026.11463454

Closest papers

Nearest neighbours by meaning across the whole index.

Support Vector Machines Directly Minimise KL Divergence

cs.LG stat.ML ·2026-11-03 ·Ivan Silajev
similarity 0.51
Abstract · 14 pages, 1 table, v2 includes section on optimal solution

Support vector machines (SVMs) are fundamentally grounded in geometric margin maximisation and currently lack a mainstream probabilistic interpretation. This keeps them theoretically isolated from probabilistic machine learning models that are naturally comparable and combinable. In this paper, we bridge this gap by proving that the dual objectives of both support vector classification and support vector regression are exact, constrained Kullback-Leibler (KL) divergence minimisation tasks between two Gaussian measures. We further show that this same divergence formulation recovers the exact objective function for kernel ridge regression when domain constraints are removed and $L^2$ regularisation is applied. By demonstrating that SVM duals directly minimise relative entropy, our results provide a unified information-theoretic model for standard kernel machines and lay the foundation for unifying broader kernel-based algorithms under probability theory.

0.23

Revisited Support Vector Machines: A Unified Joint Optimization Framework with Nyström Approximation for Scalable Adaptive Learning

cs.LG ·2026-10-11 ·Nassou Gongon Benjamin, Noulapeu Ngaffo, Patrick Njionou Sadjang
similarity 0.48
Abstract

The theoretical elegance of Support Vector Machines (SVMs), grounded in statistical learning theory, establishes them as a reference for robust classification. However, in the era of Big Data, classical SVMs face two major bottlenecks: the prohibitive cubic complexity $\mathcal{O}(N^3)$ of training and the rigidity of static kernels. Traditional methodologies treat kernel selection as a discrete, "brute-force" GridSearch problem, effectively decoupling the learning of data representation from the optimization of the decision boundary. This paper proposes a paradigm shift with a so-called "Revisited SVM", a unified framework that recasts the learning process as a continuous %bi-level joint optimization problem. We derive a novel objective function $J(α, θ)$ that allows for the simultaneous optimization of dual Lagrange multipliers and kernel hyper-parameters. To solve this challenging non-convex problem efficiently, we introduce a hybrid Block Coordinate Ascent (BCA) algorithm, tightly integrated with Nyström-based low-rank approximation, which drastically reduces computational complexity to $\mathcal{O}(Nm)$. Extensive experiments conducted across four heterogeneous domains (Bioinformatics, Finance, Cybersecurity, and Computer Vision) demonstrate that our approach not only matches but often exceeds exhaustive GridSearch accuracy, while delivering training speedups of up to 100x. Notably, on the complex NSL-KDD intrusion detection dataset, our adaptive kernel approach achieves 96.25\% accuracy compared to a mere 50\% for standard baselines, proving its unique ability to model complex non-linear manifolds in high-dimensional spaces. We validate our results using rigorous statistical tests and provide a theoretical convergence analysis of the proposed algorithm.

0.18

Point-Supervised Visual Learning for GNSS Measurement Quality Assessment

eess.SP ·2026-10-10 ·Weiwei Zhai, Penggao Yan, Liang Wang, Li-Ta Hsu
similarity 0.47
Abstract

Multipath and non-line-of-sight (NLOS) propagation degrade global navigation satellite system (GNSS) positioning accuracy in urban environments. Although fisheye sky segmentation identifies obstructions along satellite directions, binary visibility does not fully characterize observation quality. This study proposes a point-supervised visual framework that predicts GNSS observation faults from fisheye images using supervision derived from GNSS observations. During offline labeling, a vision-assisted differential GNSS procedure selects multiple reference satellites based on sky visibility and elevation constraints. Double-difference pseudorange residuals automatically generate fault labels and sample importance weights at satellite projection locations. A residual-weighted loss is computed only at valid labeled points, enabling learning from sparse supervision without manual pixelwise fault annotations. The trained network generates dense fault probability maps, which are sampled at satellite projection locations to weight observations in weighted least squares positioning, reducing the influence of potentially faulty observations. The framework is evaluated using U-Net, DeepLabv3+, DINOv3, and SAM3 on urban vehicle data collected on a different day from the training data. All four models reduce horizontal root mean square error (RMSE) by 29.51% to 36.07% compared with SAM binary sky segmentation, with U-Net achieving the lowest RMSE of 0.78 m. The framework also provides positioning solutions during some intervals when the segmentation baseline fails. These results demonstrate the framework's generalization ability and show that GNSS-derived point supervision enables fisheye images to provide observation quality information beyond binary sky visibility, supporting GNSS fault detection.

0.32

Tracking the Invisible: Self-Supervised Wheel Track Detection with Thermal-RGB Fusion

cs.CV cs.RO ·2026-10-09 ·Yiming Yang, Jeremy P. Bos
publishedcode/data similarity 0.47
Abstract · IROS 2026, 8 pages, 6 figures

Accurate detection of drivable area is essential for autonomous driving and advanced driver assistance systems. While state-of-the-art methods focus on general traversability estimation, they prove inadequate for snow-covered roads where following existing wheel tracks offers crucial advantages: improved traction, predictable vehicle dynamics, and verified path stability. To address this, we propose a low-cost, self-supervised approach detecting safety-critical wheel tracks using synchronized RGB-I (RGB + Long-Wave Infrared) imagery. Our method automatically generates training labels by projecting future wheel trajectories from recorded GNSS data. A neural network is designed to identify wheel tracks with the 4-channel RGB-I images from challenging, white-out road surfaces. Experimental results demonstrate the effectiveness of our approach in visually monotonous and featureless adverse environment, offering a robust and cost-effective alternative for winter autonomy. The newly introduced ThermalTrack dataset is available at: https://digitalcommons.mtu.edu/thermaltrack/.

0.42

Kernel-Based Metrics Learning for Uncertain Opponent Vehicle Trajectory Prediction in Autonomous Racing

cs.RO cs.AIcs.LG ·2026-09-15 ·Hojin Lee, Youngim Nam, Sanghun Lee, Cheolhyeon Kwon
top venuecode/data similarity 0.47
Abstract · Accepted version of the article published in IEEE Robotics and Automation Letters

Autonomous racing confronts significant challenges in safely overtaking Opponent Vehicles (OVs) that exhibit uncertain trajectories, stemming from unknown driving policies. To address these challenges, this study proposes heterogeneous kernel metrics for Deep Kernel Learning (DKL), designed to robustly capture the diverse driving policies of OVs, and carry out precise trajectory predictions along with the associated uncertainties. A key virtue of the proposed kernel metrics lies in their ability to align similar driving policies and disjoin dissimilar ones in an unsupervised manner, given the observed interactions between the Ego Vehicle (EV) and OVs. The efficacy of the proposed method is substantiated through experimental studies on a 1/10th scale racecar platform, demonstrating improved prediction accuracy and thereby safely overtaking against OVs. Furthermore, our method is computationally efficient for onboard computing units, affirming its viability in fast-paced racing environments. The video and source code can be found at https://github.com/HMCL-UNIST/OpponentPredictionWithKMDKL.git.

0.80

Kernel Singular Value Decomposition with Extension to Multiple Data Sources

cs.LG ·2026-10-02 ·Xinjie Zeng, Qinghua Tao, Johan Suykens
theory similarity 0.47
Abstract

Kernel Singular Value Decomposition (KSVD) learns a pair of singular vectors w.r.t. an asymmetric kernel matrix, which can be induced by two data sources, e.g., the queries and keys in self-attention or the rows and columns of a given matrix. In this work, we extend KSVD to multiple data sources, namely eKSVD, which conducts joint nonlinear feature learning upon asymmetric kernels. In the primal formulation, the projections associated with each data source are jointly learned to capture maximal information, while incorporating pair-wise couplings. With the Lagrangian and its Karush-Kuhn-Tucker (KKT) conditions, the optimization in the dual leads to a generalization of the shifted eigenvalue problem in Lanczos decomposition theorem of KSVD. Further, a covariance-based framework is derived together with using neural networks (NNs) for explicit feature mappings, complementary to the kernel-based interpretation and optimization. Numerical experiments verify the effectiveness of our eKSVD compared to methods based on Mercer kernels for tackling multiple data sources, and our innovation of deploying NNs demonstrates great flexibility for kernel methods.

0.21

Sparse and robust geometric twin support vector machine via asymmetric RoBoSS loss function

cs.LG ·2026-08-12 ·Kai Qi, Xinji Huang, Hongchun Wang
similarity 0.47
Abstract

In real-world scenarios, the training data usually contains redundant features, label noise and feature noise, which provide severe challenges for the efficiency of machine learning methods. Since standard support vector machine (SVM) adopts $l_2$-norm penalty and hinge loss function, it lacks the ability of selecting significant features and is sensitive to noise. To address these issues, this paper proposes a novel asymmetric, robust, bounded, sparse and smooth (aR) loss function for $l_1$-norm penalized geometric twin SVM (aRSGTSVM) to handle classification and regression tasks. The $l_1$-norm penalty can achieve the feature selection. The proposed aR loss function can not only effectively mitigate the impact of label noise, but also significantly enhance the stability to resampling noise, i.e., the zero-mean feature noise around the boundary hyperplanes. Furthermore, a statistical analysis of the robustness of aRSGTSVM was also conducted using the influence function. Since aRSGTSVM involves nonconvex and nonsmooth optimization, we develop a fast and stable proximal gradient descent based solving algorithm. Compared with related state-of-the-art methods, experimental results demonstrate the superiority of the proposed aRSGTSVM on both synthetic and UCI datasets. Furthermore, we apply aRSGTSVM to index tracking tasks, where results for tracking the different indices in the China stock market show that it can achieve satisfactory performance.

0.04

ReinSeg: Retrieval-Guided Bounded Multimodal Refinement for 3-D Road Anomaly Segmentation

cs.CV ·2026-11-02 ·Haifa Zhang, Yijing Wang, Zhiqiang Zuo
benchmark similarity 0.46
Abstract

3-D road anomaly segmentation is essential for risk-aware perception in autonomous driving. Its central challenge is to rank sparsely observed hazards ahead of abundant normal road structures while preserving point-wise metric localization. LiDAR provides accurate 3-D point support, whereas surround-view imagery offers complementary semantic context for separating unexpected hazards from challenging normal structures. We propose ReinSeg, a retrieval-guided bounded multimodal refinement framework for point-wise road anomaly segmentation. Occupied LiDAR bird's-eye-view cells retrieve scene-specific context from a shared memory of eight surround-view images without supplying calibration variables to the retrieval module. A relative-norm-bounded residual returns this context to the LiDAR point features, so the frozen LIDO anomaly-scoring interface is preserved, and the refinement is optimized with in-distribution semantic labels only. On the Spotting the Unexpected (STU) benchmark, ReinSeg improves validation average precision from 27.54\% to 35.67\% and reduces the false-positive rate at 95\% anomaly recall from 34.85\% to 10.61\%, shrinking the high-recall screening set from 67.54 to 20.55 million points. Relative to REL, AP is 24.99 percentage points higher and FPR95 is 0.99 points lower. Matched interfaces and component isolation show that retrieved scene imagery and the relative-norm bound are both essential, and ReinSeg stays above LiDAR-only performance under missing, corrupted, or misaligned camera inputs while running at 21.8 frames per second. These results establish bounded retrieval of scene-specific visual evidence as an effective route to reliable 3-D road-hazard perception.

0.19