CS Paper Radar

NVE: A Separability and Coverage-Aware Internal Validation Metric for Biclustering

Paritosh Tiwari, I Navin Kumar, James C. Bezdek, Punit Rathore

cs.LG ·submitted 2026-08-29 04:35 UTC · v1
unusual

Abstract

Biclustering, or co-clustering, aims to discover coherent submatrices by grouping rows and columns of a data matrix simultaneously. This local two-dimensional structure makes validation more difficult than in ordinary clustering, where internal indices usually rely on compactness and separation in a single shared feature space. Existing popular internal biclustering measures such as Mean Squared Residue (MSR), and Virtual Error (VE) mainly evaluate within-bicluster coherence. Although useful, these measures do not directly assess whether the extracted biclusters are mutually distinct or whether they explain a meaningful portion of the data matrix. This paper investigates Normalised Virtual Error (NVE), an internal validation metric that extends VE using a super-bicluster normalization strategy. By comparing the VE of each bicluster with the VE obtained after merging it with other biclusters, NVE introduces a relative notion of separability and redundancy. We also study a coverage-adjusted variant, NVE\textsubscript{cov}, which penalizes solutions that obtain low error by selecting only very small submatrices. Through controlled synthetic benchmarks and yeast gene-expression datasets, we examine whether NVE and NVE\textsubscript{cov} provide information beyond standard coherence-based metrics. The results show that NVE is sensitive to redundant and poorly separated biclusters, while NVE\textsubscript{cov} changes solution rankings when low-error biclusters cover only a negligible part of the matrix. These findings suggest that NVE-based measures are useful complementary criteria for internal co-clustering validation, especially when coherence, separability, and coverage must be considered jointly.

Closest papers

Nearest neighbours by meaning across the whole index.

Federated Clustering with Unknown Local and Global Cluster Cardinalities

cs.LG cs.DCstat.ML ·2026-09-29 ·Mitushi Goyal, Tarun S., Riddhanya Senapathi, Arun Raman
similarity 0.55
Abstract

Federated clustering methods that do not require the global number of clusters $K$ still assume that each client knows its local number $K_g$. This assumption is hard to justify when clients know no more about their data than the server does, as in fault diagnosis across independently operated industrial sites. We propose a two-phase framework in which neither count is known: each client first estimates $K_g$ from its own data, and an aggregator that requires local counts, such as FedGEM, then uses these estimates in place of the true values. For the first phase we introduce Adaptive Split--Merge (ASM), which grows a spherical Gaussian mixture by BIC-driven splitting and then merges excess components. ASM uses no labels, selects its hyperparameters on held-out client data only, and makes no assumption about how clusters are shared across clients. We derive a closed-form split criterion whose critical cluster size falls with anisotropy and rises with dimension, and show empirically that over-fragmentation grows with the number of points per cluster, which federation divides among clients. Across eight datasets, ASM with FedGEM attains a mean ARI of 0.333, against 0.256 for the next best label-free estimator and 0.361 when the true local counts are supplied. It also gives the most reliable global estimates of $K$ and is robust when client size is decoupled from local cardinality.

0.23

Selective Inference for Clustering Under Unknown Covariance

cs.LG stat.MEstat.AP ·2026-10-12 ·Yiqun T. Chen, Lucy L. Gao
similarity 0.53
Abstract

Unsupervised clustering is widely used to discover underlying group structure, after which researchers often seek to validate these groups via post-hoc hypothesis testing on the population means. However, using the same data both to form clusters and to test for a difference in means inflates the Type I error rate. Recent work in selective inference has proposed solutions when the underlying data are drawn from a multivariate Gaussian distribution with a known covariance matrix or an unknown isotropic covariance matrix. We propose a new selective test for the difference in means between two clusters that provably controls the Type I error rate under an arbitrary unknown covariance matrix. Our test statistic does not require estimating the feature covariance matrix; instead, it leverages a Monte Carlo sampling procedure over the set of orthogonal matrices. We show that this new selective test can be computed efficiently, and we derive closed-form solutions when clusters are formed using k-means or hierarchical clustering. In simulations, we demonstrate that our proposed p-value controls the selective Type I error rate and achieves substantial power, and we illustrate its application on gene expression and drug review data sets.

0.22

A precision-driven evaluation protocol for author name disambiguation

cs.DL ·2026-10-09 ·Francesca Cappelli, Giovanni Colavizza, Silvio Peroni
code/data similarity 0.51
Abstract

Author Name Disambiguation systems are commonly evaluated using pairwise and B$^3$ metrics, which summarize clustering quality through aggregate precision and recall but do not explicitly distinguish between lumping (merging publications from different authors) and splitting (fragmenting the publications of a single author). Because these errors can have asymmetric consequences in bibliographic applications, we propose a precision-driven evaluation framework that decomposes clustering quality into complementary instance-level and cluster-level signals. The framework introduces cluster-level Lumping Error and Splitting Error to separately quantify contamination and fragmentation, combines them with standard pairwise and B$^3$ metrics through a Composite Score, and uses the resulting measures for model selection. We characterize the diagnostic behavior of the framework through a controlled synthetic case study and assess the sensitivity of the Composite Score to its weighting scheme. The results show that the proposed structural measures distinguish error patterns that aggregate metrics can mask, while the sensitivity analysis identifies a stable region for the adopted weighting configuration. The framework is released as an open-source Python pipeline.

0.42

Randomized SVD Approximations for Spectral Co-Clustering of Word-Document Matrices

cs.LG cs.AI ·2026-09-16 ·Fateme Mazdarani, Carlos Toxtli
published similarity 0.50
Abstract · Accepted to 2026 IEEE International Conference on Machine Learning and Applications (ICMLA)

Spectral co-clustering is a useful tool for discovering latent structure in word-document matrices, but its reliance on singular value decomposition (SVD) can make standard formulations expensive on high-dimensional data. This paper presents two randomized approximations for normalized spectral co-clustering of bipartite text data when the numbers of document and word clusters may differ. The first method uses randomized SVD through random projection, while the second combines partial SVD with element-wise random sampling. Across real-world and synthetic datasets, both methods reduce runtime relative to the full-SVD baseline, but their behavior depends on matrix sparsity. The random projection method is the more reliable approximation across the tested settings, whereas the sampling-based method is most useful on denser matrices and provides limited benefit on already sparse text data. These results show that randomized approximations for spectral co-clustering should be selected according to the underlying structure of the data.

0.31

Resolving Multi-Modal Regression by Difference-Quotient-Based Clustering:Fast Coarse Conditional-Label Assignment

cs.LG ·2026-08-26 ·Huang Weiquan
similarity 0.49
Abstract

Multimodal regression suffers from the mean-collapse pathology: under squared loss, an unconstrained regressor converges to the conditional mean, which for K > 1 lies away from all modes. We attribute this failure to pairwise contradictions--samples with nearly identical inputs but distant outputs--and propose Difference-Quotient Clustering (DQC), which partitions data to minimize intra-cluster output-vs-input discrepancy. Each sample is assigned to the cluster that minimizes its maximum contradiction ratio; a logits generator and a conditional network are then trained on the resulting labels. Since the generating modality is unknown at test time, we evaluate via minimum squared error (minMSE) against all K true outputs. On synthetic benchmarks (K=5, 10), DQC achieves test minMSE 0.19 (K=5, nx=500), versus 0.09 for an oracle, 1.08 for random labels, and 1.33 for mean collapse. We observe two empirical regularities: larger intra-cluster contradictions require deeper networks, and oracle labels generalize from fewer samples than cluster-derived equivalents. The clustering is a hard, parallelizable O(n^2/2) front-end for coarse conditional assignment, reducing the burden of downstream generative refinement. A second-stage re-clustering on residual errors is outlined as future work.

0.28

Matrix-Aware Proper Scoring Rules and Significance Testing for Correlation and Covariance Forecasts in Python

cs.MS stat.ME ·2026-08-30 ·Vinh Nguyen
published similarity 0.49
Abstract

Forecasting a correlation or covariance matrix is common in risk management and portfolio construction, but evaluating such a forecast correctly is not routine: naive matrix-comparison metrics are not proper scoring rules, walk-forward evaluation windows are easy to overlap with the estimation window in ways that silently leak information, and significance testing on serially dependent forecast-error sequences needs machinery few analysts implement from scratch. corrscore is a Python package that provides matrix-aware implementations of two established proper scoring rules for this setting -- the energy score and the variogram score -- dispatched across a closed-form tractability spectrum (point, discrete-mixture, and isotropic-Gaussian-mixture forecasts are scored exactly; a general Monte Carlo ensemble falls back to sampling), a geometry-aware variant of the variogram score built from the affine-invariant distance on the correlation manifold, a zero-overlap-by-construction walk-forward backtest harness, and a bundled significance-testing suite (circular block bootstrap, the Diebold-Mariano test, and the Model Confidence Set). We describe the package's design, its point of departure from the existing scoringRules and properscoring packages, and walk through a complete worked example.

0.36

PathFinder: Joint Decompositions of Linked Multimodal Datasets

cs.LG eess.IVq-bio.QMstat.ML ·2026-08-15 ·Ying-Qiu Zheng, Alex Fung, Stephen M Smith, Rogier B Mars, Saad Jbabdi
similarity 0.48
Abstract

Low-rank matrix decompositions can uncover patterns and structure in data and have a number of different applications across many disciplines. Extensions to "joint" low-rank decompositions have been proposed to link datasets from different modalities. While these methods enable the discovery of common patterns across modalities, they require that all the multimodal data share one or more dimensions. We propose a new analysis method, PathFinder, that enables co-analysis of datasets that do not necessarily all share a dimension. The key insight is that as long as pairs or subgroups of matrices do share some dimension, and that there are one or more paths that link across the data matrices, a global joint decomposition can be sought out. This enables the joint estimation of common patterns across different modalities, species, or scales, where a one-to-one mapping across all data along some dimension is not necessarily available. We show that PathFinder is a general umbrella under which many matrix decomposition methods fall as special cases. It can be used to discover common patterns across disparate datasets and to make predictions for missing data or modalities.

0.04

An Approach to Study the Structural Consistency of Triangle Badness Functions and Distance Metrics

cs.CG math.MGstat.AP ·2026-08-24 ·Bowen Liu, Yizhou Wang, Lingqian Meng
similarity 0.48
Abstract · 18 pages, 11 tables. Main text in English; includes computational experiments on biological, astronomical, and materials datasets

Triangle-based measures, commonly referred to as badness functions, are widely employed to quantify the extent to which a distance matrix deviates from an ideal geometric configuration. Different formulations of these functions may capture distinct facets of local non-uniformity, and their behavior is often influenced by the underlying distance metric chosen for evaluation. In practical settings, although a canonical badness function may be conceptually preferred, factors such as computational cost, algorithmic constraints, or data-specific characteristics frequently necessitate the adoption of modified versions-for instance, approximate forms or alternatives defined under different distance metrics. This gives rise to a central question: to what degree do these variants retain the structural consistency properties of their original counterparts? To address this issue, we develop a systematic correlation-based framework for evaluating structural consistency. As an illustrative instantiation of this framework, we compute badness sequences from a set of representative distance matrices alongside randomly generated triangle configurations, which are designed to cover variants that may arise under diverse practical scenarios. We then assess pairwise similarities among these sequences using four correlation coefficients. The experimental outcomes indicate that certain badness variants exhibit a notably high degree of structural consistency, whereas others reveal complementary behavioral patterns; moreover, the choice of distance metric exerts a considerable influence on the observed trends. These findings offer practical insights for the informed selection of distance metrics and triangle badness function variants in tasks including geometric reconstruction, triangulation, and structural analysis of pairwise distance data.

0.31