CS Paper Radar

Causal Input--Output Modeling with Prior Information

Amirhossein Taghvaei, Daran Xu

eess.SY math.OC ·submitted 2026-11-17 00:31 UTC · v1
theory

Abstract

We study the problem of inferring a causal input--output model from unpaired marginal observations and a given prior coupling. We formulate this problem using entropic causal optimal transport, which seeks a coupling that preserves the observed marginals, satisfies a one--sided causality constraint, and remains close to the prior. For Gaussian marginals and general quadratic costs, we prove that the optimal coupling is Gaussian and reduce the problem to a finite--dimensional optimization over covariance matrices. Using a Cholesky transformation to simplify the causality constraint, we derive a Lagrangian dual formulation and develop a block coordinate descent algorithm with explicit update formulas. We establish linear convergence of the algorithm, with explicit convergence rates, and prove stability of the optimal coupling under marginal covariance perturbations and derive a statistical error bound. We further show its equivalence to Sinkhorn iterations. Numerical experiments demonstrate the computational efficiency of the proposed algorithm and illustrate the predictive advantages of causal couplings.

Closest papers

Nearest neighbours by meaning across the whole index.

Optimal input design via Frank-Wolfe

eess.SY math.OC ·2026-09-08 ·Fethi Bencherki, Bruce Lee, Nikolai Matni, Anders Rantzer
similarity 0.64
Abstract · 7 pages, 4 figures

We study optimal input design over a finite horizon for linear dynamical systems. The goal is to minimize a weighted inverse-covariance (information) criterion subject to an energy budget. The set of covariances achievable by causal policies is convex but lacks a tractable explicit description, ruling out projection-based methods. We show that Frank--Wolfe applies naturally: each linear minimization subproblem is a budget-constrained finite-horizon linear quadratic (LQ) problem, solvable by a Riccati recursion and one-dimensional bisection over a Lagrange multiplier. Using smoothness of the objective over the feasible set, we establish an $\mathcal{O}(1/M)$ convergence rate for the objective value, while strong convexity yields an $\mathcal{O}(1/\sqrt{M})$ rate for the iterates. We further extend the framework to input design for system identification with unknown dynamics and adaptive online LQR, and illustrate the approach numerically.

0.28

Efficient SCA Algorithms for Sparse Bayesian Learning

eess.SP ·2026-10-26 ·Esa Ollila, Majdoddin Esfandiari, Daniel P. Palomar, Xinjue Wang
theory similarity 0.61
Abstract

Type-II sparse Bayesian learning (SBL) for the multiple measurement vector model recovers a sparse signal power vector by minimizing the marginal negative log-likelihood via the expectation-maximization (EM) algorithm, referred to as M-SBL. We revisit the covariance learning (CL) problem and develop three parallel first-order algorithms within the successive convex approximation (SCA) framework. CL-SCA minimizes per-coordinate surrogates formed by partially linearizing the log-determinant term, while CL-FP uses a damped projected fixed-point iteration derived from stationarity conditions. Both methods have closed-form updates. We prove that the fixed points of both update maps are exactly the KKT points of the CL problem, that each update direction satisfies a quantitative descent inequality, that the damped iterates achieve sufficient decrease with an $\mathcal{O}(T^{-1/2})$ best-iterate stationarity rate, and we derive a backtracking rule that retains this rate without step-size tuning. If damping factors are bounded away from zero, the residual vanishes along the whole sequence and every limit point is a KKT point. Local analysis motivates CL-Hyb, which combines CL-SCA and CL-FP. Experiments on device activity detection in massive random access show that CL-Hyb-bt matches the best CL solvers' accuracy at a fraction of the runtime and is over an order of magnitude faster than EM.

0.44

Transporting the Past: An Optimal Transport View of Backtracking Counterfactuals

cs.AI cs.LG ·2026-11-05 ·Zhangyi Liu, Pouria Fatemi, Suvrit Sra
similarity 0.60
Abstract

We study backtracking counterfactual inference for Structural Causal Models (SCMs) through the lens of optimal transport. In SCMs, a factual observation generally induces a posterior distribution over exogenous variables, making counterfactual inference inherently distributional. We introduce two complementary frameworks that transport this posterior onto the set of exogenous configurations satisfying a desired counterfactual query. The first constructs a natural pointwise projection and is suitable when the transported exogenous distribution can be used directly. The second enforces mutual independence among counterfactual exogenous variables, so that the transported variables remain interpretable as genuine exogenous noises of the same SCM. We show that both frameworks are consistent with the original backtracking counterfactual principle, and we propose marginal, penalty-based, and hard-constrained solvers for finite-posterior settings. Synthetic experiments on non-bijective SCMs illustrate the geometry of the problem and reveal trade-offs among transport cost, counterfactual constraint satisfaction, and preservation of exogenous independence.

0.18

Differentiable Structure Learning for Cyclic Linear Gaussian Models with Latent Confounders

cs.LG cs.AI ·2026-09-29 ·Sadegh Khorasani, Ali Najar, Saber Salehkaleybar, Negar Kiyavash
theory similarity 0.60
Abstract · 42 pages, including appendices

We study causal structure learning from observational data in linear Gaussian structural causal models in the presence of directed cycles and an unknown number of exogenous latent confounders, bounded by a given maximum. We derive the covariance of the observed variables and introduce marginal quasi-equivalence, which characterizes when different causal models share a full-dimensional subset of the observational distributions they can generate. We formulate structure learning as minimization of the Gaussian negative log-likelihood with a logarithmically scaled complexity penalty that counts directed edges and latent variables. For a fixed number of observed variables and a fixed upper bound on latent variables, we establish consistency of global score minimizers up to marginal quasi-equivalence under algebraic faithfulness, structural minimality, and model-overlap assumptions. We parameterize the inclusion of directed edges and candidate latent variables using Bernoulli gates, whose continuous probabilities are optimized jointly with the structural coefficients. Averaging the penalized negative log-likelihood over these gates yields an objective with a closed-form differentiable complexity penalty. We prove that this expected objective has the same global infimum as the corresponding discrete structure-learning objective. Experimental results show that our approach achieves lower recovery error than previous methods in several experimental settings.

0.27

Derivative Gaussian Processes on a Two-Direction Budget

cs.LG stat.ML ·2026-10-07 ·Hyunseok Seung, Matthias Katzfuss
similarity 0.59
Abstract

Gradient observations promise more accurate Gaussian process (GP) surrogates, but the cost of incorporating them has long stood in the way of realizing that promise. We propose a derivative GP with a budget of just two directions per observed gradient. One direction focuses on each gradient's direct contribution to target prediction, while the other aggregates its indirect contributions through correlations with the conditioning function values. Within a Vecchia approximation, where each prediction conditions on $m$ nearby inputs in $d$ dimensions, this construction represents their $md$ gradient coordinates using at most $2m$ directional derivatives, giving $\mathcal{O}(m^3)$ dense factorization cost per prediction target. For general conditioning sets, we bound the posterior approximation error relative to using full gradients and characterize when the error is small or the approximation is exact. In simulations, our method matches the accuracy of a leading exact gradient-reduction method at equal conditioning set size. Because its cost grows much more slowly with that size, it can use conditioning sets well beyond the memory limit of the exact method, reaching lower prediction error with a small fraction of the time and memory. Notably, our method can exploit gradient observations while requiring less computation time or memory than function-only GP baselines.

0.29

The Statistical Cost of Causal Discovery with Feedback

cs.LG stat.ML ·2026-09-28 ·Sunmin Oh, Seungsu Han, Gunwoong Park
similarity 0.57
Abstract

What determines the unavoidable sample cost of learning cyclic causal structure? For cyclic linear non-Gaussian models, we study exact condensation recovery from observational data: identifying the strongly connected component (SCC) partition and all edges between components. We establish the first information-theoretic lower bounds on sample complexity for this target. For $p$ variables, maximum SCC size $s_{\max}$, and maximum external-parent count $d_B$, any estimator requires order $s_{\max}\log(ep/s_{\max})+d_B\log(ep/d_B)$ samples in the worst case over a regular model class. These bounds distinguish the costs of SCC membership and external-parent selection. Under principal invertibility and without correlation faithfulness, we establish a population block-exogeneity principle that identifies unknown root SCCs through residual independence and inclusion minimality. A sparse-adjustment characterization shows that small adjustment sets suffice to identify SCCs and their direct external parents, without regressing on all previously recovered variables. These characterizations yield BlockExo, which attains a structurally matching sample bound without knowing $s_{\max}$ or $d_B$ under suitable conditions. Simulations support the structural dependence of our sample bound and demonstrate BlockExo's sample-efficient recovery in comparisons with other methods for cyclic causal discovery.

0.26

Calibrated Derivative-Process Sensitivity for Gaussian-Process Variable Selection

cs.LG stat.MLmath.NA ·2026-09-27 ·Jia Cai
hot direction similarity 0.57
Abstract

Automatic relevance determination (ARD), the default tool for variable selection in Gaussian-process (GP) regression, ranks inputs by inverse lengthscales -- which measure how fast a function varies, not how much an input contributes to prediction -- and offers no calibrated rule for deciding which inputs to keep. The prediction-centred alternative, the derivative sensitivity $ν_j = \mathbb{E}[(\partial f/\partial x_j)^2]$, is available in closed form from a fitted GP, but turning it into a selection rule is harder than it looks: at a null input the estimator is a degenerate quadratic form, so Wald and Bernstein-von Mises cutoffs are anti-conservative, and the natural residual bootstrap is mis-scaled. We show that a studentized multiplier bootstrap of the GP derivative process repairs both, prove its validity through an invariance principle for quadratic forms, and obtain asymptotic family-wise and false-discovery-rate control across inputs. Over 100 replications the rule controls FDR wherever inputs are truly null, while uncalibrated derivative rankings breach the target by up to 2x and a Bernstein-von Mises cutoff by 2.2x; at matched FDR it loses no power; it holds under a Matérn kernel and input correlation up to 0.99; on real data with planted and authentic null inputs it admits 5-12x fewer spurious inputs; it costs 5-18% of the GP fit; and a block-averaged variant retains validity at cost linear in $n$.

0.35

Causal Discovery in Equal Variance Linear Gaussian DAGs via SURE-Tuned Ridge Regression

cs.LG eess.SPstat.ML ·2026-08-17 ·Sambit Mishra, Urbashi Mitra
published similarity 0.57
Abstract · 5 Pages, 3 Figures. Accepted at 60th Asilomar Conference on Signals, Systems, and Computers 2026

Recovering the directed acyclic graph (DAG) of a structural equation model (SEM) from observational data is a central problem in causal discovery. The iterative gradient descent and per-problem hyperparameter tuning of continuous-optimization methods are poorly suited to two practically important regimes: the sample-limited regime, where the number of samples is comparable to or smaller than the number of nodes in the DAG, and the compute-limited regime. This work proposes SURE-Ridge, a non-iterative, closed-form estimator for equal variance linear Gaussian SEM. The method performs parallel node-wise regressions with regularization parameters chosen adaptively by Stein's unbiased risk estimate (SURE), and applies an adaptive thresholding procedure to extract a DAG from the resulting soft adjacency matrix. Numerical results show that SURE-Ridge achieves the lowest structural Hamming distance in the small-sample regime and the lowest run time across all sample sizes tested, compared with NOTEARS, DAGMA, and GBNSL baselines.

0.57