CS Paper Radar

chATLAS: A Retrieval-Augmented Conversational Assistant for the ATLAS Experiment

Joe Egan, Ben Elliot, Samuel Van Stroud, Alex Sopio, Ki Ryeong Park, Daniele Dal Santo, Zhaohui Li, Runze Li, Jeremy Couthures, Joe George, Cary Randazzo, Juerg Beringer, Daniel Murnane, Gabriel Facini

cs.IR hep-ex ·submitted 2026-10-24 12:51 UTC · v1
benchmark

Abstract

Large scientific collaborations produce extensive internal documentation spread across heterogeneous systems, making it difficult for collaborators to locate and reuse existing knowledge. We present chATLAS, a deployed Retrieval-Augmented Generation (RAG) assistant developed for the ATLAS experiment at CERN. The system provides conversational access to more than 30,000 documents from several collaboration resources, material largely unavailable to general-purpose language models, using hybrid dense--sparse retrieval deployed on CERN infrastructure. A domain-specific benchmark built from automatically generated question--answer pairs is used to evaluate retrieval and generation performance and to guide design choices. We study the selection and domain-specific fine-tuning of an embedding model, contextual retrieval, ranking strategies, and generator choice. Fine-tuning combined with contextual retrieval more than doubles how often the correct document is returned as the top result on the most technically challenging documentation source. The deployed system also includes intent routing, safety handling, Mattermost integration, and an MCP server for future agentic workflows. chATLAS demonstrates that domain-adapted RAG can provide source-attributed, domain-specific access to collaboration knowledge, and offers a practical foundation for scientific collaborations and other organisations that rely on accurate, up-to-date retrieval from proprietary documentation.

Comment
20 pages, 6 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Rethinking Knowledge Retrieval for Generation: A Survey on RAG Architectures and Applications

cs.AI ·2026-10-01 ·Meghana Sunil, Shravya V, Shravan Venkatraman, Joe Dhanith PR
survey similarity 0.74
Abstract

Large Language Models (LLMs) have demonstrated remarkable fluency and versatility across natural language tasks but remain fundamentally limited by their static knowledge and susceptibility to hallucinations, especially in domains requiring up to date or attribute grounded information. Retrieval Augmented Generation (RAG) addresses these challenges by integrating external retrieval mechanisms with generative models, enabling dynamic, context aware generation grounded in verifiable data sources. This survey presents a comprehensive examination of RAG as a modular and evolving paradigm that enhances factual reliability, adaptability, and task alignment in LLM based systems. We formalize the RAG framework through its three foundational components retrieval, generation, and augmentation and survey state of the art methods spanning dense and sparse retrievers, fusion strategies, embedding optimizations, and reinforcement learning based retrieval policies. Anchored around four emerging axes efficiency, security, user centric interactivity, and complex reasoning we categorize recent innovations and highlight their implications for scalability, robustness, and personalization. The paper also reviews advances in evaluation protocols, domain specific applications, and architectural variants such as Naïve RAG, Advanced RAG, and Modular RAG. Finally, we identify persistent challenges and outline future directions aimed at advancing the integration of retrieval with LLMs for more grounded, interpretable, and controllable generation.

0.07

Re-ranking and Late Interaction Drive Retrieval Quality: A Controlled Comparison of RAG Strategies for Scientific Question Answering

cs.IR cs.AIcs.LG ·2026-09-29 ·Bhagyesh Rathi, Eshan Chawla, William B. Andreopoulos
similarity 0.73
Abstract · on September 21st submitted for consideration to the Elsevier Data and Information Management (DIM) journal (DIM-D-26-00430)

Retrieval-Augmented Generation (RAG) is now the standard way to ground Large Language Models (LLMs) in external knowledge, yet the design space of retrieval pipelines is large and the trade-offs between variants are not well understood, especially on domain-specific corpora at realistic scale. In this work, we present a controlled comparison of six retrieval strategies for scientific question answering: (i) classic top-k dense retrieval, (ii) LLM-based query rephrasing, (iii) query rephrasing followed by LLM-based reranking, (iv) multi-query fusion via Reciprocal Rank Fusion (RRF), (v) an agentic tool-call pipeline in which the generator decides for itself whether to retrieve, and (vi) late-interaction retrieval with ColBERTv2. All six pipelines share the same generator (Meta-Llama/Llama-3.1-8B-Instruct), prompt, and evaluation protocol; the five single-vector pipelines additionally share SPECTER2 embeddings and a Chroma vector store; and all six retrieve from the full corpus of 463,971 arXiv papers dated 2024-2025. To support reproducible, large-scale evaluation, we also release a synthetic question dataset of 19,484 problem-statement and methodology questions generated by Llama-3.1-8B-Instruct from a random sample of 10,000 papers across academic domains (query generation succeeded for 9,742 of them), and every strategy is evaluated on this same query set. We describe the architecture and implementation of each pipeline, release the code and the synthetic question dataset, and evaluate each strategy with an LLM-as-a-judge protocol along multiple quality dimensions, together with direct gold-paper retrieval metrics. The result is an open testbed for studying the cost and quality trade-offs of RAG design choices on a research-literature corpus, and a basis for future work on faithfulness, retrieval robustness, and agentic retrieval.

0.12

Retrieval Augmented Generation Compositionality

cs.CL cs.AI ·2026-10-21 ·Ido Levy, Segev Shlomov, Leshem Choshen
code/datacommunity pick similarity 0.73
Abstract · 103 pages (9 main), 15 figures, 68 tables. Project page: https://dolev31.github.io/lasagna/ Code: https://github.com/dolev31/lasagna

Retrieval Augmented Generation (RAG) grounds large language models in external knowledge. Modern pipelines combine embedding, chunking, retrieval, reranking, query augmentation, context compression, and agentic verification, yet these techniques are typically evaluated one at a time. It remains unclear whether their isolated gains survive composition, whether retrieval quality and answer faithfulness favor the same design choices, and whether conclusions from small corpora hold at large scale. We pre-declare three hypotheses and introduce LASAGNA, an open-source framework for controlled RAG evaluation. We test 450 configurations across eight benchmarks and corpora ranging from 5K to 200K documents. Three findings emerge. First, individually strong techniques from different phases of a RAG pipeline do not necessarily compose: three retrieval techniques predict a +47% combined improvement but deliver only +17% when used together. Second, component gains stack differently for retrieval and faithfulness, and optimizing one can hurt the other. Third, the best composition changes as the corpus grows: the combination that wins on a small corpus may not remain best at large scale. These findings show that gains measured in isolation are not reliable predictors of realistic performance, and that new RAG techniques should be tested with the full surrounding stack. We call on researchers to do so and supply LASAGNA to simplify these experiments.

0.44

EGT-KG: Evidence-Grounded Typed KG Retrieval for Practical Scientific QA with Small Language Models

cs.AI ·2026-08-31 ·Muran Yu, Jiechao Gao, Yuandong Pan, Barney H. Miao, Andrew C. Lesh +3 more
top venuebenchmark similarity 0.73
Abstract · Accepted in EMNLP Industry track 2026

For emerging scientific research domains, local Small Language Models (SLMs) are becoming more attractive, as they offer stronger privacy control and more stable deployment pipelines than Large Language Models. However, in practice, scientific question-answering on SLMs often operates under inevitable constraints: small literature collections, fragmented evidence, limited context window and reasoning abilities. We propose the Evidence-Grounded Typed Knowledge Graph (EGT-KG), a retrieval framework to improve information retrieval with local SLMs. We assessed three question-answering settings: a vanilla Retrieval-Augmented Generation (RAG) workflow and two EGT-KG workflows: an automatically generated relation schema (AS) and an expert-defined relation schema (ES). Our experiments were evaluated with a six-dimensional evaluation framework (S3CRF: Soundness, Correctness, Completeness, Conciseness, Relevance, Fluency) on a Biopolymer-bound Soil Composite literature benchmark, showing that EGT-KG outperforms the vanilla RAG method in most settings, with the best improvement from llama3:8b: a Final Score of 70.37 (+14.67%) and 68.82 (+12.14%) by AS/ES EGT-KG variants.

0.45

From Chunks to Functional Evidence: Function-Aware Retrieval for EDA Documentation QA

cs.IR cs.AI ·2026-10-07 ·Xiaotian Qiu, Kairui Liu, Shi Chenyi, Jinyuan Deng, Qi Sun +1 more
benchmark similarity 0.73
Abstract · 10 pages, 2 figures, including appendices

Retrieval-Augmented Generation (RAG) is widely used to ground answers in documents. For complex technical documentation, however, the primary bottleneck is often not model reasoning but a mismatch between a query and the way knowledge is organized for retrieval. This mismatch is pronounced in Electronic Design Automation (EDA) documentation, where the information needed for an answer is scattered across heterogeneous yet tightly coupled artifacts. We therefore redesign the basic retrieval unit of RAG. Instead of operating on isolated chunks or binary relations, we collect typed artifacts into EDA functional units. Each unit is recorded as a hyperedge with links to its source chunks. We then train an encoder to align queries with functional units and combine unit retrieval with direct chunk retrieval. After mapping the selected units back to their sources, a unified reranker chooses the evidence given to the generator. On the newly constructed EDADocEval-QA dataset, our method improves ROUGE-L by 37.1% over Chunk RAG and 55.6% over the strongest graph baseline. On the public ORD-MMBench benchmark, it improves ROUGE-L by 30.0% over the strongest baseline. These results support function-aware evidence organization in the evaluated EDA documentation settings.

0.18

RAGit: Turning Document Preprocessing into an Experimental Variable for RAG

cs.IR ·2026-10-17 ·Omar El Bachyr, Saad Ezzini, Tegawendé F. Bissyandé, Jacques Klein
publishedcode/data similarity 0.72
Abstract · 4 pages, 2 figures, 4 tables. Accepted at the Tool Demonstration Track of the 33rd Asia-Pacific Software Engineering Conference (APSEC 2026)

Document preprocessing choices can substantially affect retrieval in Retrieval-Augmented Generation (RAG) pipelines, yet they are often selected heuristically and are difficult to compare under consistent conditions. We present RAGit, an open-source experimentation workbench for the systematic and reproducible evaluation of document preprocessing choices through downstream retrieval. RAGit represents parsing, chunking, indexing, retrieval, and evaluation as configurable experimental stages, enabling controlled comparisons while the remaining downstream settings are kept fixed. It preserves provenance from retrieved chunks to source pages and maps heterogeneous retrieval units to a common page-level ranking for evaluation. The current implementation supports multiple parsers and chunking strategies, dense and late-interaction retrieval, reusable intermediate artifacts, and multi-configuration benchmarking. We demonstrate RAGit through controlled experiments comparing alternative preprocessing configurations under a fixed retrieval setup, illustrating how preprocessing can be treated as an explicit experimental variable. RAGit and its documentation are publicly available on GitHub (https://github.com/OmarElbachyr/ragit). A demonstration video is available at https://youtu.be/tsst-Bhfz4I.

0.29

RATIO: A Benchmark for Retrieval Across Typed Ideation Operations in Scientific Literature

cs.CL cs.IR ·2026-08-27 ·Maayan Sharon, Tom Hope
benchmark similarity 0.71
Abstract

Retrieved scientific literature can serve as inspiration for both human and AI scientists. Inspiration can take different forms: prior work may directly suggest how to address a problem, or surface directions at different levels of abstraction---zooming out to a more general view or zooming in to a concrete realization. We introduce RATIO (Retrieval Across Typed Ideation Operations), a large-scale benchmark in which relevance is defined by three operations which we name ideation moves: Address retrieves potential approaches for stated problems, Broaden retrieves more general formulations, and Specify retrieves concrete instantiations. RATIO is constructed from millions of full-text scientific papers across CS literature via a general recipe that extends discourse-marker distant supervision---previously used only for classification---to corpus-scale retrieval, combined with extensive LLM and human vetting. Experiments show that operation-specific fine-tuning substantially boosts retrievers and LLM reranking helps further, but much room for improvement remains. Researchers blind to condition found the fine-tuned retriever's candidates more useful as inspirations. RATIO provides a scalable training and evaluation framework for retrieval components that support literature-grounded ideation, opening up new research avenues in scientific inspiration retrieval. We release full data, code, models, and annotations.

0.16

W-RAG: Source-Aware Retrieval for Enterprise Document Generation from Heterogeneous Knowledge Bases

cs.SE cs.CLcs.IR ·2026-08-22 ·Hridya Dhulipala, Rajesh Ombase, Michael Wang, Tien N. Nguyen
dataset similarity 0.71
Abstract · 18 pages

Retrieval-Augmented Generation (RAG) enables large language models to incorporate external knowledge during generation, improving factual grounding and domain adaptability. However, existing RAG pipelines assume that evidence retrieved from multiple repositories can be ranked globally using a single similarity function. While suitable for open-domain retrieval, this assumption breaks down in enterprise document generation, where heterogeneous knowledge bases (such as policies, regulations, technical documentation, and departmental guidelines) serve distinct roles and must be jointly represented in the generated document. As a result, global ranking often produces unbalanced context dominated by a subset of sources, leading to incomplete enterprise drafts. To address this limitation, we propose W-RAG, a source-aware retrieval framework that performs ontology-guided retrieval, local ranking within each knowledge base, and source-level weighting to regulate evidence composition. We further introduce a new dataset for retrieval-grounded enterprise document generation spanning multiple document types and industry domains. Experiments show that standard RAG pipelines struggle on this task, while W-RAG significantly improves document coverage and generation quality.

0.25