CS Paper Radar

TopicMerge: A Divide and Conquer Approach to Topic Modeling

Matthew Raffel, Lizhong Chen, Sai Munikoti, Sameera Horawalavithana

cs.CL cs.LG ·submitted 2026-10-14 00:01 UTC · v1
top venue

Abstract

Recent advancements in topic modeling have split into two main paths: efficient embedding-based methods that offer scalability but are often outperformed on standard clustering metrics, and Large Language Model (LLM) approaches that achieve state-of-the-art clustering performance but at the cost of prohibitive computational expense and poor scalability. This paper introduces TopicMerge, a novel multi-stage pipeline that bridges these approaches to achieve both state-of-the-art clustering performance and computational efficiency. TopicMerge first employs an efficient clustering pipeline to deliberately over-segment a document corpus into a large set of fine-grained, semantically pure micro-clusters, which are then given initial semantic labels. It then introduces a novel, iterative merging algorithm where an LLM, guided by local neighborhood context, arbitrates on consolidating candidate pairs. These approved merges are then selectively executed and semantically relabeled by the LLM, hierarchically building broader, more coherent topics. Extensive experiments on the Bills and Wiki datasets show that TopicMerge achieves state-of-the-art clustering performance while reducing required LLM calls by over 92% compared to document-level methods, offering a practical and scalable solution.

Comment
Findings of EMNLP 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Modeling Document and Section Hierarchies in Embedding Space

cs.AI stat.MLcs.CLcs.LG ·2026-11-16 ·Kriste Krstovski, Sharyn O'Halloran, Kshitij Chandna
similarity 0.73
Abstract

Embedded topic models integrate word embeddings into the topic generation process, producing coherent topics and robust performance on large vocabularies. However, each document is represented by a single topic distribution, ignoring its internal structure. Many real world collections, including legal documents, company and government reports, and online discussion forums, exhibit hierarchical organization in which documents consist of multiple sections that often address distinct subtopics. We introduce the Hierarchical Embedded Topic Model (hETM), a structured topic model that couples the topic distribution of a document and those of its sections through a hierarchical prior in the embedding space. We derive an amortized variational inference procedure for hETM and evaluate it on structured collections from each of these domains. Experimental results show that hETM achieves the lowest or near-lowest held-out perplexity among flat and hierarchical topic model baselines, along with comparable or higher topic quality. It also performs strongly on a downstream clustering task, achieving the best clustering performance on most of the collections.

0.10

Similar but Not Redundant: Embedding-Gated Refinement for High-Granularity Topic Models

cs.CL cs.IR ·2026-11-12 ·Daniel Denley, Robert L. Nicol, Mike J. Chantler
position similarity 0.72
Abstract · 24 pages, 6 figures, 13 tables

High-granularity topic modelling extracts hundreds of niche topics from a corpus, yet this regime is rarely studied and is more commonly framed as a failure mode: "over-clustering," in which coherent themes fragment into redundant subtopics. We argue that this framing conflates genuine redundancy with topics that are semantically similar but meaningfully distinct, and that the central challenge in high-granularity modelling is separating the two. Recent post-hoc refinement methods address topic redundancy through either embedding-based similarity thresholding or LLM-guided merging. We show that neither suffices in high-granularity settings and propose embedding-gated iterative merge (EGIM), a hybrid approach that iteratively generates context-rich topic representations, uses embedding similarity to propose candidate merges across the topic space, then applies LLM judgement to accept only genuinely redundant pairs while preserving niche distinctions. Across two patent corpora and one legal corpus, LLM judgement rejects 83% of embedding-identified candidates as non-redundant. EGIM improves discriminability on merged topics by 25-55% over a prior iterative LLM baseline while better preserving the original topic distribution.

0.14

Sentence-Level Topic Modeling with Lexical and Contextual Evidence

cs.CL cs.LG ·2026-10-16 ·Federica Granese, Serena Villata, Charles Bouveyron
similarity 0.66
Abstract

Topic modeling faces a fundamental granularity dilemma: probabilistic topic models assign topics at the word level, whereas transformer-based methods operate at the document level. The latter overlooks that real-world documents typically cover multiple topics, whereas the former relies on individual words, which rarely provide sufficient context to identify them. We introduce SentenceETM, a neural topic model that assigns a topic to each sentence by jointly using lexical and contextual evidence. Each topic is described both by a distribution over the vocabulary, defined through word embeddings, and by a Gaussian over sentence embeddings concentrated on a topic-specific low-dimensional subspace. The topic of a sentence generates both its words and its embedding and is marginalized exactly, so that its posterior combines the two sources of evidence. Document proportions are inferred with amortized variational inference, while the Gaussian parameters are re-estimated in closed form. Experiments show that SentenceETM consistently improves sentence-level topic assignment over state-of-the-art approaches, and a case study of a presidential debate shows how sentence-level topics reveal how candidates handle the same subjects.

0.19

Does Syntax Matter? A Graph-Augmented Variational Topic Model for Computational Social Sciences

cs.CL ·2026-09-07 ·Alessandro Meneghini
similarity 0.65
Abstract · 33 pages, 1 figure

Topic modeling is widely used in computational social sciences to identify latent themes in large text corpora. Traditional approaches rely on Bag-of-Words representations and generative models such as LDA, while recent methods like BERTopic operate on dense document embeddings. This paper introduces the Structural Contextual Probabilistic Topic Model (SCPTM), an architecture that incorporates syntactic dependency relations into topic inference. SCPTM represents a corpus as a heterogeneous graph of documents and words connected by lexical and syntactic edges, processed through a Graph Attention Network within a Variational Autoencoder to produce probabilistic, mixed-membership topic distributions. We evaluate seven topic modeling techniques (including four SCPTM ablations) across four corpora differing in register and discourse structure. Our framework combines coherence (C_V, C_NPMI), topic diversity, clustering-label alignment (NMI), and phrase-level diagnostics (complementarity and valence gap). Results show that SCPTM's neural architecture yields substantial gains in document-topic alignment over generative baselines, but these gains are attributable to the variational encoder rather than to syntax. Syntax contributes to topic diversity, where graph-augmented variants outperform the no-graph baseline across all corpora, and to descriptor quality: dependency paths capture predicate-argument structures and stance in deliberative registers, while proving redundant in technical and institutional corpora. The valence gap is positive across all variants, but driven primarily by phrase grouping rather than syntactic filtering. We conclude that syntactic encoding matters conditionally: it benefits action-oriented, argumentative texts, but introduces noise in informational or administrative registers.

0.20

Beyond Top Words: MonoTM for Topic Modeling with Interpretable Monosemantic Features

cs.CL cs.HC ·2026-09-09 ·Una Joh, Bei Yu
publishedbenchmark similarity 0.65
Abstract · Accepted to appear in the Proceedings of AACL-IJCNLP 2026

Topic models summarize large text corpora, but top-ranked words often provide only a limited representation of topic semantics. Sparse autoencoders (SAEs) offer a way to move beyond word-level descriptors by extracting interpretable features from dense representations, yet how feature interpretability relates to topic-inference quality remains unclear. We introduce \textbf{MonoTM}, an interpretable topic modeling framework that decouples these roles. Across three benchmark corpora, we show that document--topic mixture estimation and semantic interpretation favor different SAE configurations and feature subsets. MonoTM estimates mixtures from the full SAE bag-of-features representation and, with them fixed, learns topic descriptors over a separate vocabulary of corpus-grounded semantic features. This design preserves global topic structure while representing topics with semantic units more meaningful than individual words, making them more useful for downstream corpus analysis.

0.50

Segment-Level Agentic Topic Modeling for Improved Data Exploration and Resource Efficiency

cs.AI ·2026-09-25 ·Myeongjun Erik Jang, Antonios Georgiadis, Sae Young Moon, Fran Silavong
similarity 0.65
Abstract

Topic modeling is an effective technique for discovering hidden themes within documents and is widely used in text mining and data analysis across a variety of industry sectors. Recently, large language model (LLM)-based topic models have been emerged that prompt LLMs to generate topics then assign the topics to documents, producing more natural and human-readable topics than conventional topic modeling algorithms. However, the nature of topic assignment process causes certain drawbacks, such as the incapability to produce topic distributions over a document, too broad or narrow topics, and high resource consumption, which increases with the number and length of of documents being assigned topics. These issues are particularly critical for industrial applications, which require high-quality, in-depth analysis and the processing of large volumes of documents. In this context, this paper introduces a framework called SeLATM, which addresses these concerns by employing segment-level topic generation and topic refinement through agentic feedback loops. Experimental results on various datasets demonstrate that SeLATM significantly reduces the LLM resources compared to methods based on topic assignment process, while maintaining superior performance.

0.20

LEDGER: LLM-Evidence-Driven Graph-Encoded Clustering-Friendly Representations for Short Texts

cs.SI ·2026-11-13 ·Jie Wang, Kuang Zhou
similarity 0.65
Abstract

Short-text clustering is important for analyzing user-generated content in social networks and online services, supporting applications such as topic discovery and user intent analysis. However, the desired grouping depends on an application-specific clustering criterion that general-purpose text embeddings may not capture. Large language models (LLMs) can apply such a criterion to a pair of texts without human annotation, and existing methods use their responses as generated descriptions, binary decisions or selected neighbours. These discrete outputs discard how strongly the LLM supports a pair and how many judgments stand behind it. Each judgment is also used alone or within the neighbourhood of a single text, so the relations that link judgments across the corpus remain unused. In this paper, we propose LEDGER, an LLM-guided graph representation learning framework for short-text clustering. LEDGER aggregates the repeated numerical scores of each text pair into an accumulated score and an observation count, which define an evidence target for every queried pair. The pairs whose scores reach a threshold form a relation graph, over which a graph encoder propagates text features with a propagation range learned jointly with the encoder. A pairwise predictor trained to match the evidence targets shapes the learned representation, and a distilled student encoder extends it to unqueried texts without further LLM requests. Extensive experiments on seven datasets with four frozen text embeddings show that LEDGER achieves the highest mean clustering accuracy and NMI against five comparison methods, and ablations show that the graph, the learned propagation range and the evidence target each improve performance.

0.10

EviMap: Evidence-Grounded Hierarchical Topic Maps for Exploring Unlabeled Corpora

cs.IR cs.CLcs.HC ·2026-09-06 ·Zhiyin Tan, Changxu Duan
top venuesurveycode/datahot direction similarity 0.63
Abstract · Accepted at the 35th ACM International Conference on Information and Knowledge Management (CIKM '26)

Research teams and organizations often explore unfamiliar free-text collections, from survey comments and reviews to reports and domain documents, before labels, queries or coding schemes exist. At this stage, the first thematic map shapes what users notice, prioritize and carry into downstream analysis, so it should be trusted only insofar as it can be verified. Existing options force a trade-off between scale and verifiability. Qualitative coding preserves evidence but is slow. Search presupposes a query. Clustering and topic models scale but produce labels users must interpret. One-shot large language model (LLM) summaries are fluent yet difficult to reproduce or audit. We present EviMap, an interactive system providing researchers and practitioners with an auditable thematic overview of such corpora. Guided by model-generated context describing the corpus and hypothesized stakeholder concerns, EviMap extracts within-document evidence phrases and organizes them, rather than whole documents, into a three-level map of aspects, groups and fine-grained topics. Embedding-based clustering narrows the search space for finer semantic judgments by the LLM. Each node traces back to supporting phrase spans, so documents link to topics through evidence they contain and users can audit labels against the original text. Users can start from a top-level corpus map, drill into topics, inspect highlighted evidence in original documents, and combine two topics to find documents discussing both. We demonstrate this workflow across six heterogeneous corpora spanning 2,108 to 101,699 documents, with a comparison against flat and hierarchical LLM baselines. By grounding every label in verbatim source spans, EviMap makes a topic map not just readable, but verifiable. Code, demo video, and interactive dashboard are available at https://github.com/zhiyintan/EviMap.

0.60