CS Paper Radar

EviMap: Evidence-Grounded Hierarchical Topic Maps for Exploring Unlabeled Corpora

Zhiyin Tan, Changxu Duan

cs.IR cs.CLcs.HC ·submitted 2026-09-06 15:14 UTC · v1
top venuesurveycode/datahot direction

Abstract

Research teams and organizations often explore unfamiliar free-text collections, from survey comments and reviews to reports and domain documents, before labels, queries or coding schemes exist. At this stage, the first thematic map shapes what users notice, prioritize and carry into downstream analysis, so it should be trusted only insofar as it can be verified. Existing options force a trade-off between scale and verifiability. Qualitative coding preserves evidence but is slow. Search presupposes a query. Clustering and topic models scale but produce labels users must interpret. One-shot large language model (LLM) summaries are fluent yet difficult to reproduce or audit. We present EviMap, an interactive system providing researchers and practitioners with an auditable thematic overview of such corpora. Guided by model-generated context describing the corpus and hypothesized stakeholder concerns, EviMap extracts within-document evidence phrases and organizes them, rather than whole documents, into a three-level map of aspects, groups and fine-grained topics. Embedding-based clustering narrows the search space for finer semantic judgments by the LLM. Each node traces back to supporting phrase spans, so documents link to topics through evidence they contain and users can audit labels against the original text. Users can start from a top-level corpus map, drill into topics, inspect highlighted evidence in original documents, and combine two topics to find documents discussing both. We demonstrate this workflow across six heterogeneous corpora spanning 2,108 to 101,699 documents, with a comparison against flat and hierarchical LLM baselines. By grounding every label in verbatim source spans, EviMap makes a topic map not just readable, but verifiable. Code, demo video, and interactive dashboard are available at https://github.com/zhiyintan/EviMap.

Comment
Accepted at the 35th ACM International Conference on Information and Knowledge Management (CIKM '26)
DOI
10.1145/3799682.3840239

Closest papers

Nearest neighbours by meaning across the whole index.

TaxCE : A Framework for Automated Taxonomy Construction and Evaluation at Scale

cs.CL ·2026-08-31 ·Sandeep Sricharan Mukku, Albert Aristotle Nanda, Rohit Pyati
top venue similarity 0.66
Abstract · EMNLP 2026 - Industry

Organizing unstructured feedback text into hierarchical taxonomy is a fundamental challenge in NLP, particularly in domains where feedback arrives at massive scale in varied forms such as reviews, transcripts, and surveys. Existing approaches either produce shallow hierarchies, neglect long-tail topics, or lack rigorous evaluation frameworks. We present TaxCE, a fully automated framework that constructs multi-level hierarchical taxonomies from raw text through progressive condensation of corpus content into actionable segments, deduplicated semantic units, and granular topics with definitions, which are then organized bottom-up into a hierarchy with corpus-groundedness. We also introduce three corpus-grounded evaluation metrics, Exclusivity, Exhaustivity, and Granularity (EEG), and integrate them into a metrics-in-the-loop iterative refinement mechanism that diagnoses deficiencies and applies targeted corrections until convergence. Extensive experiments demonstrate that TaxCE consistently outperforms existing baselines spanning classical topic models, neural methods, and LLM-based approaches, with average improvements of 11.8, 20.5, and 15.7 percentage points in exclusivity, exhaustivity, and granularity respectively over the strongest baseline. Human evaluation further confirms superior taxonomy quality, actionability, and navigability.

0.51

TractorBeam: Personalized AI Sensemaking Support via Collaborative Machine Annotation

cs.HC ·2026-08-19 ·Sireesh Gururaja, Jordan Taylor, Emma Strubell
top venue similarity 0.66
Abstract · UIST Poster Extended Abstract

Language model-based systems which allow asking questions of documents have become popular tools for sensemaking. Despite their implied capability, these systems still suffer from issues of factuality and provenance, while encouraging confirmatory, rather than exploratory, research. We present TractorBeam, a browser extension-based mixed-initiative system that uses collaborative annotation as an interface metaphor for sensemaking, re-framing language model (LM) outputs as suggested highlights in a process that we call collaborative machine annotation. This metaphor allows us to present LM results in-context on PDF documents, directly addressing concerns of provenance and factuality, while allowing users to iteratively construct mental schemas and queries for language models directly in the context of a document. In a preliminary user study, all of our participants felt that TractorBeam enabled them evaluate and iteratively improve the model's reflection of their intended highlighting, and several found suggestions that made them reconsider their original schema. TractorBeam suggests that systems that facilitate exploratory research on individual documents may lead to verifiable sensemaking for users and complement tools that work across broader corpora.

0.31

Modeling Document and Section Hierarchies in Embedding Space

cs.AI stat.MLcs.CLcs.LG ·2026-11-16 ·Kriste Krstovski, Sharyn O'Halloran, Kshitij Chandna
similarity 0.66
Abstract

Embedded topic models integrate word embeddings into the topic generation process, producing coherent topics and robust performance on large vocabularies. However, each document is represented by a single topic distribution, ignoring its internal structure. Many real world collections, including legal documents, company and government reports, and online discussion forums, exhibit hierarchical organization in which documents consist of multiple sections that often address distinct subtopics. We introduce the Hierarchical Embedded Topic Model (hETM), a structured topic model that couples the topic distribution of a document and those of its sections through a hierarchical prior in the embedding space. We derive an amortized variational inference procedure for hETM and evaluate it on structured collections from each of these domains. Experimental results show that hETM achieves the lowest or near-lowest held-out perplexity among flat and hierarchical topic model baselines, along with comparable or higher topic quality. It also performs strongly on a downstream clustering task, achieving the best clustering performance on most of the collections.

0.10

LEDGER: LLM-Evidence-Driven Graph-Encoded Clustering-Friendly Representations for Short Texts

cs.SI ·2026-11-13 ·Jie Wang, Kuang Zhou
similarity 0.65
Abstract

Short-text clustering is important for analyzing user-generated content in social networks and online services, supporting applications such as topic discovery and user intent analysis. However, the desired grouping depends on an application-specific clustering criterion that general-purpose text embeddings may not capture. Large language models (LLMs) can apply such a criterion to a pair of texts without human annotation, and existing methods use their responses as generated descriptions, binary decisions or selected neighbours. These discrete outputs discard how strongly the LLM supports a pair and how many judgments stand behind it. Each judgment is also used alone or within the neighbourhood of a single text, so the relations that link judgments across the corpus remain unused. In this paper, we propose LEDGER, an LLM-guided graph representation learning framework for short-text clustering. LEDGER aggregates the repeated numerical scores of each text pair into an accumulated score and an observation count, which define an evidence target for every queried pair. The pairs whose scores reach a threshold form a relation graph, over which a graph encoder propagates text features with a propagation range learned jointly with the encoder. A pairwise predictor trained to match the evidence targets shapes the learned representation, and a distilled student encoder extends it to unqueried texts without further LLM requests. Extensive experiments on seven datasets with four frozen text embeddings show that LEDGER achieves the highest mean clustering accuracy and NMI against five comparison methods, and ablations show that the graph, the learned propagation range and the evidence target each improve performance.

0.10

Self-prompting and cross-model consensus enable reproducible data extraction from scientific literature with large language models

cs.AI cs.DB ·2026-08-19 ·Valentin Romanov, Monique Bax, Steven Niederer
similarity 0.64
Abstract

Accurately extracting nuanced, contextualized data from research articles is laborious and time intensive. Here, we investigate the performance of frontier, browser-based large language models (LLMs) to extract highly contextualized information. We demonstrate four escalating workflows, 1) given an expert curated prompt and research articles, most frontier LLMs perform well at data extraction, however can struggle with interpreting scientific context and nuance, 2) given simple instructions, LLMs can author their own prompts which were almost as eNective as expert-written prompts, 3) autonomous discovery of research literature was diNicult, agents either missed or hallucinated references, and 4) LLMs can create new datasets from published guidelines that closely match human-expert judges, but still require a human-in-the-loop. Together, these findings define an auditable division of labour in which experts specify the evidence standard, models cross-check repeated extractions and researchers resolve disputed cases, providing a practical route to scaling scientific data curation without relinquishing expert oversight.

0.18

SGHA: Evidence-Grounded Research Problem Discovery with Local Language Models

cs.AI cs.LG ·2026-08-18 ·Sarvesh Gharat, Junpei Komiyama
code/data similarity 0.64
Abstract · Link to Code and artifacts: https://github.com/SarveshVGharat/structural-gap-hypothesis-agent

Recent efforts toward fully automated AI scientists have demonstrated that language-model agents can generate hypotheses, execute experiments, and draft scientific manuscripts. However, during the early stages of research, when research problems are formulated, these AI scientists often rely heavily on proprietary frontier models. Their proposals are shaped by opaque parametric knowledge and by literature searches conditioned on the proposals themselves. Such knowledge is effectively a black box, and this dependence makes the evidential basis and validity of generated research problems difficult to audit and leaves the process vulnerable to model-specific hallucinations and biases. Furthermore, if proprietary research materials are transmitted to external APIs, the use of these models creates confidentiality, privacy, and data-governance concerns. We introduce the Structural Gap Hypothesis Agent (SGHA), a fully automated, corpus-first research-problem discovery system that runs entirely on a local LLM. SGHA structures a scientific literature corpus into evidence-linked paper objects and a typed evidence graph, detects unresolved structural patterns across papers, screens candidate gaps before formulation, and produces traceable research-problem families. In particular, it is able to output assumptions, objectives, success criteria, and remaining ambiguities. All LLM-based components of SGHA are executed using a locally served open-weight 9B language model, without requiring proprietary frontier-model APIs. We compare SGHA with the AI Scientist-v2 idea formulation module in five machine-learning domains. Our results suggest that explicit corpus structure and evidence-constrained reasoning can support promising, inspectable research-problem formulation without relying on frontier models during generation or verification.

0.36

FootprintRAG: Visual Analytics for Evidence Context Refinement in RAG-based Scientific Literature Exploration

cs.GR cs.HCcs.IR ·2026-09-17 ·Xingyu Liu, Yu Dong, Qizhen Yu, Shiyu Cheng, Zhe Wang +5 more
code/data similarity 0.63
Abstract

Retrieval-Augmented Generation (RAG) is increasingly used to ground large language model (LLM) outputs in scientific literature. However, in open-ended literature exploration, the evidence context used for generation is often produced through hidden retrieval, reranking, assessment, and filtering steps. Users may receive retrieval summaries without knowing how the system constructed the evidence context, which evidence units were retained or discarded, or whether potentially useful evidence was excluded before synthesis. We present FootprintRAG, an LLM-agent-powered visual analytics system for evidence context refinement in RAG-based scientific literature exploration. The core idea is to treat the RAG evidence context as an explicit, inspectable, and revisable analytical object before generation. FootprintRAG parses scientific literature into text and figure evidence units, expands an initial query into parallel query variants, retrieves and assesses evidence across iterative rounds, and surfaces ERS-ranked supplementary candidates from the corpus-level evidence space. Through coordinated views, the system connects retrieval trajectories, evidence-state revision, and provenance-aware summary generation into a user-steerable workflow. We evaluate FootprintRAG through two case studies, a user study, and a workflow-level comparison with representative RAG systems. The results show that FootprintRAG helps users compare retrieval directions, revise candidate evidence, recover potentially overlooked evidence, and trace generated summaries back to supporting evidence units. FootprintRAG is available at https://github.com/meteorshowering/FootprintRAGVA.git.

0.24

Intent Graph: Navigating the Analytical Reasoning Space for Exploratory Data Analysis

cs.HC cs.AI ·2026-10-08 ·Junran Yang, Shruti Badrish, Teanna Barrett, Leilani Battle
similarity 0.63
Abstract

Exploratory data analysis (EDA) is rarely open-ended in practice: analysts work from high-level domain questions toward the concrete analyses that can answer them, prioritizing directions with domain knowledge and prior hypotheses. Large language models (LLMs) can supply such knowledge, but their responses are unstructured, leaving analysts no way to see what has been explored, what is missing, or why one direction was chosen over another. We present DAG-EDA, a system that lets analysts and an LLM co-navigate the space of possible analyses through two linked structures. An intent graph, governed by a grammar of analytical intent, decomposes an ambiguous natural-language question into progressively concrete analysis tasks, keeping alternative framings open and letting analysts branch, backtrack, and compare paths. A multi-layered knowledge graph externalizes the LLM's domain knowledge, linking domain concepts to the dataset variables that can measure them, so analysts can inspect and contest how their question is grounded in the data. Both graphs are constructed from only the dataset and the analyst's question, and the analyses the analyst reaches are rendered as interactive dashboards. We illustrate the system through a usage scenario and describe a user study design for examining whether the system scaffold analysts' reasoning and navigation.

0.07