CS Paper Radar

Hessian Rank Constraint for Learning Structure of Nonlinear Latent Variable Models

Zijian Li, Ruichu Cai, Feng Xie, Xinshuai Dong, Haoyue Dai, Yuewen Sun, Yujia Zheng, Guangyi Chen, Yingyao Hu, Kun Zhang

cs.LG cs.AI ·submitted 2026-09-21 08:05 UTC · v1

Abstract

Uncovering latent variables and their causal relations from observed data is a fundamental yet challenging problem. Existing methods often rely on restrictive assumptions, such as linear relations or invertible mixing functions. To better address this problem under general nonlinear mixing procedures, we propose a condition called the cross-Hessian Rank Constraint (HRC), which serves as a primitive rank-based tool for nonlinear latent causal discovery. In particular, we show that a rank-based property arises from the cross-Hessian of the observed-data log-density in the nonlinear case, revealing information about the latent variables, and reduces to the Tetrad constraints in the linear Gaussian case. More specifically, when two groups of observed variables are d-separated by a set of lower-dimensional latent variables, the rank of this cross-Hessian is equal to the dimension of the latent variables, under a mild affine derivative assumption on the conditional log-density derivatives. This assumption can be naturally satisfied when the noise level is low or the relevant nonlinearity is moderate. As a downstream application, we instantiate HRC in the pure one-factor measurement setting for locating latent variables and recovering their causal structure up to Markov equivalence. Experimental results on synthetic and real-world datasets support the theoretical claims.

Closest papers

Nearest neighbours by meaning across the whole index.

Differentiable Structure Learning for Cyclic Linear Gaussian Models with Latent Confounders

cs.LG cs.AI ·2026-09-29 ·Sadegh Khorasani, Ali Najar, Saber Salehkaleybar, Negar Kiyavash
theory similarity 0.62
Abstract · 42 pages, including appendices

We study causal structure learning from observational data in linear Gaussian structural causal models in the presence of directed cycles and an unknown number of exogenous latent confounders, bounded by a given maximum. We derive the covariance of the observed variables and introduce marginal quasi-equivalence, which characterizes when different causal models share a full-dimensional subset of the observational distributions they can generate. We formulate structure learning as minimization of the Gaussian negative log-likelihood with a logarithmically scaled complexity penalty that counts directed edges and latent variables. For a fixed number of observed variables and a fixed upper bound on latent variables, we establish consistency of global score minimizers up to marginal quasi-equivalence under algebraic faithfulness, structural minimality, and model-overlap assumptions. We parameterize the inclusion of directed edges and candidate latent variables using Bernoulli gates, whose continuous probabilities are optimized jointly with the structural coefficients. Averaging the penalized negative log-likelihood over these gates yields an objective with a closed-form differentiable complexity penalty. We prove that this expected objective has the same global infimum as the corresponding discrete structure-learning objective. Experimental results show that our approach achieves lower recovery error than previous methods in several experimental settings.

0.27

Provable Guarantees and Efficient Learning of Structural Equation Models with Latent Confounders

cs.LG stat.ML ·2026-09-16 ·Weijian Yu, Jean Honorio
similarity 0.61
Abstract

Causal discovery aims to recover causal relationships from observed data. In various fields, exploring causal relationships among variables remains an important topic, but this task becomes challenging due to the existence of latent confounders. Ignoring such confounders can lead to false associations and incorrect edge directions. In this paper, we study the linear structural equation model with latent confounders. We propose an algorithm that iteratively identifies terminal (observed) nodes and reconstructs the directed acyclic graph of the observed variables. To do this, we recover the precision matrix of the observed variables as a sparse plus low-rank matrix: a sparse matrix captures the conditional dependencies among observed variables, while a low-rank matrix captures the combined influence of a few latent confounders. We establish that for $p$ observed variables, $r$ latent confounders and $s$ edges, our procedure correctly identifies the directed causal relationship among observed variables, for $n \gtrsim \max\{s\log p,\ r p\}$ samples. Experimental results validate our theoretical contributions.

0.20

Causal Discovery for Linear Models with Latent Confounding via Multi-domain Identifiable Representation Learning

cs.LG cs.AI ·2026-10-20 ·Danru Xu, Aapo Hyvärinen, Sara Magliacane
similarity 0.58
Abstract · 29 pages, 6 figures

Causal discovery with latent confounding is particularly challenging, but a common setting in real-world data. In this work, we focus on linear causal models with latent confounding, represented by acyclic directed mixed graphs (ADMGs). Existing methods that fully identify ADMGs typically assume non-Gaussianity or graphical constraints, such as bow-freeness. In contrast, we propose Linear ADMG Learning (LinADMG), a framework for learning ADMGs for linear causal models through identifiable representation learning. Rather than relying on non-Gaussianity or bow-freeness, LinADMG leverages multi-domain settings to identify the noise variables. We show that the identifiability of the noise variables is sufficient for recovering all direct causal relations, and that, under an additional latent-confounding no-cancellation condition, LinADMG can identify the whole ADMG. We implement LinADMG with four identifiable representation learning approaches from the literature and validate the methods on simulated and semisynthetic datasets.

0.30

Proximal Balancing for Causal Effect Estimation under Unmeasured Confounding

cs.LG stat.MLstat.ME ·2026-09-30 ·Yonghan Jung
code/data similarity 0.58
Abstract · 59 pages. Code: https://github.com/CausalDataScience/proximal-balancing

Estimating causal effects from observational data is central to science and policy, but the effects are not identified when confounders are unmeasured. Proximal causal inference addresses this problem with proxies of the unmeasured confounders. However, existing proxy-based approaches either designate proxy roles and solve an inverse problem, which is ill-posed and hard to estimate with high-dimensional proxies, or use a latent-variable model, which assumes that the learned latent variable matches the hidden confounder and leaves bias when it does not. To address these challenges, we introduce proximal balancing. It carries the classical idea of covariate balancing to confounders that are observed only through proxies: it learns a low-dimensional summary of the covariates and proxies that makes the treatment groups comparable, and then adjusts for this summary. It needs no designated proxy roles, inverse problem, or latent model. We give identification theory, finite-sample guarantees, and a practical algorithm, PROBE. We demonstrate the method on low-dimensional, high-dimensional, and image proxies and on real-world data.

0.33

Model-Based Learning of Latent Euclidean Embedding from Relational Data

cs.LG stat.ML ·2026-10-31 ·Yinghang Chen, Yunxiao Chen, Haoran Zhang
similarity 0.57
Abstract

Visualizing and interpreting the dependence structure in relational data often benefits from geometric representations via low-dimensional Euclidean embeddings. However, existing approaches are typically restricted to specific response types, rely on burdensome high-dimensional nonconvex optimization, or lack uniform theoretical guarantees for simultaneous coordinate recovery. To address these limitations, we introduce a unified, distance-based latent space framework for both unipartite and bipartite relational data---accommodating continuous, binary, ordinal, and count-valued observations---via a modular two-stage procedure that decouples signal estimation from geometric recovery. First, we estimate the low-rank signal matrix using spectral denoising. For models with nonlinear link functions, we further apply one-step row- and column-wise refinement to achieve uniform entrywise accuracy. Second, we recover the latent configurations through direct spectral analysis for unipartite data and a low-dimensional alignment optimization for bipartite data. We establish uniform recovery guarantees that characterize the propagation of error from signal estimation to the embedding space. To illustrate the scalability and interpretability of our approach, we apply the proposed method to two real-world datasets: a statistician citation network and European Parliament voting records.

0.30

Towards Identifiable Representations under Misspecified Structure

cs.LG ·2026-09-27 ·Yuke Li, Yujia Zheng, Ziyi Chen, Kun Zhang, Heng Huang
theory similarity 0.57
Abstract

The presence of noise that depends on the latent variables poses a fundamental challenge to identifiability. Existing results rely on conditional independence among the observations given the latent variables. We study a more general \emph{misspecified structure}, where this conditional factorization does not hold, and establish both precise and approximate identifiability guarantees. We characterize structural misspecification as a perturbed factor analysis problem. For precise identifiability, we establish subspace identifiability under spectral separation and controlled perturbation, followed by component-wise identifiability under structural sparsity. When the precise condition is not guaranteed, we derive an approximate subspace-identifiability theorem. Based on these results, we develop an unsupervised variational estimator for recovering latent variables. Experiments demonstrate the effectiveness of the proposed framework.

0.27

Jigsaw-CRL: Recovering Global Latent Causal Order from Fragmented Multi-Client Interventions

cs.LG stat.ML ·2026-08-29 ·Haijie Xu, Chen Zhang
similarity 0.57
Abstract

Causal representation learning (CRL) aims to recover latent causal variables and their structural relations from high-dimensional observations. Existing CRL methods typically assume that all environments are defined over the same latent variables, or at least share a common latent representation space. We study a fragmented multi-client setting, where multiple clients interact with the same global latent causal system but each client only accesses and intervenes on a subset of the latent variables. In this regime, marginalizing unused latent variables induces bidirected edges, so a single client no longer admits a node-wise latent causal graph, and the global latent causal order must be recovered by assembling client-specific structural fragments. We propose \textbf{Jigsaw-CRL}, a framework for recovering global latent causal order from such fragmented interventions. Under soft interventions, differences between precision matrices across environments exhibit a low-rank structure governed by latent ancestor relations. This enables recovery, for each client, of a block partition, the corresponding block-level ancestral order, and latent subspaces, and then assembly of these fragments into the global node-level latent causal order. We establish identifiability guarantees, develop practical algorithms, and validate the framework on synthetic data. Our codes are available on https://anonymous.4open.science/r/code-for-Jigsaw-CRL-7B26

0.20

The Trade-off Between Covariate Dependence and Latent Structure in Representation Learning

cs.LG ·2026-08-17 ·Małgorzata Łazęcka, Ewa Szczurek
theory similarity 0.57
Abstract

Disentangled representation learning seeks latent representations whose indicidual dimensions each align with a distinct covariate. Unsupervised approaches typically target latent dimension independence, yet this gives no guarantee that the resulting dimensions align with semantically meaningful covariates. Supervised approaches structure the latent space using observed covariates, but under correlated covariates they cannot simultaneously control one-to-one latent-covariate alignment and latent independence. We introduce a unified, supervised framework that couples latent dimension-covariate dependence with constraints on the latent structure. Within this framework, we show an inherent trade-off, where enforcing latent independence or exclusive one-to-one latent-covariate dependence comes at a provable cost in latent-covariate alignment. We prove that the resulting disentanglement regimes are ordered by the strength of that alignment. Each regime admits a closed-form transformation of the latent space. We apply these transformations post-hoc to realign the representations of pretrained models such as CLIP, DINOv2, and ViT, and we fold them into the inference of informed factor analysis (iFA), a probabilistic model with covariate-informed factors. On simulated and real multi-omics data, we show that both post-hoc alignment and iFA enable controllability of structured latent representations.

0.24