CS Paper Radar

Hybrid Retrieval for Government Scheme Search in English and Marathi

Aishwarya Gunjal, Kanchan Khemnar, Pawan Poojary, Pranav Kulkarni, Raviraj Joshi, Manish Khodaskar

cs.LG cs.CL ·submitted 2026-11-19 17:58 UTC · v1

Abstract

Retrieving government schemes requires matching citizens' information needs to formal administrative descriptions, often across languages and terminology. We present a bilingual search system for Maharashtra government schemes and public services that combines BM25 retrieval with sentence-embedding similarity through equal-weight score fusion. The system indexes aligned English-Marathi scheme descriptions and supports multilingual, Marathi-focused, and translation-mediated semantic configurations. Evaluation on manually curated bilingual queries compares BM25, dense retrieval, and hybrid retrieval using IndicSBERT and MahaSBERT. BM25 provides a strong baseline with a mean reciprocal rank (MRR) of 0.864. Hybrid retrieval increases MRR to 0.872 with IndicSBERT and 0.879 with MahaSBERT, whereas the corresponding dense-only systems achieve 0.663 and 0.655. MahaSBERT-based fusion also increases precision from 0.781 to 0.808 while matching BM25 recall at 0.946. These results support the complementary use of lexical and semantic signals in this collection, while showing that improvements over a strong lexical baseline are modest and metric-dependent. The study provides an application-focused analysis of hybrid retrieval for bilingual public-service information access.

Closest papers

Nearest neighbours by meaning across the whole index.

Query Translation vs. Cross-Lingual Embeddings for Sinhala-Tamil E-Government Information Retrieval

cs.IR ·2026-08-13 ·Dharshi Balasubramaniyam, Tiroshan Madushanka
benchmark similarity 0.77
Abstract

This paper presents a comparative evaluation of cross-lingual information retrieval (CLIR) methods for retrieving English government information using Sinhala and Tamil queries. Two CLIR paradigms are investigated: Query Translation (QT), employing Google Translate, NLLB, and mBART50, and Cross-Lingual Embeddings (CLE), using LaBSE, multilingual E5, and BGE-M3, with monolingual English retrieval as the baseline. Experiments are conducted on a human-verified benchmark comprising 500 Sinhala, Tamil, and English question-answer pairs derived from 1,699 segmented contexts from Sri Lanka's Government Information Center (GIC). Retrieval performance is evaluated using Recall@k (k = 1, 3, 5, 10, 15). Monolingual retrieval performs poorly (Recall@15 <10%), whereas all CLIR approaches substantially improve retrieval accuracy. Among them, BGE-M3 achieves the highest Recall@15, reaching 96.2% for Sinhala-English and 95.6% for Tamil-English, outperforming the best QT approach (Google Translate: 92.4% and 93.0%) while avoiding translation overhead. These results demonstrate that multilingual embedding models provide a more effective and scalable solution for cross-lingual retrieval-augmented generation (RAG) in low-resource government domains.

0.04

Where Does Retrieval Fail? Evaluating RAG Architectures for Agricultural Advisory

cs.CL ·2026-08-14 ·Khan Raiyan Ibne Reza, Sanjana Aktar Maria, Sumaiya Tabassum Nimi
code/data similarity 0.67
Abstract

Retrieval quality in RAG systems is commonly reported as a single aggregate score, which can hide large differences across query types and language conditions. We study this problem in Bengali agricultural advisory, where farmer queries are often colloquial while official advisory documents use formal scientific terminology. We construct a test collection of 1,000 queries and 2,882 knowledge nodes extracted from 284 official Bangladeshi agricultural publications, and use it to evaluate five retrieval architectures and six embedding models under three controlled language conditions. The results show that no single retrieval method is consistently best. For native Bengali queries, BM25 is the strongest single retriever (R@10 = 0.506) while Hybrid RRF reaches the highest overall R@10 of 0.539. However, dense retrieval performance varies sharply by query type: R@10 is 0.093 on colloquial farmer queries and 0.970 on formal safety queries. Across language conditions, BM25 R@10 drops from 0.506 on Bengali queries to 0.004 when English queries are matched against the Bengali corpus, while dense retrieval falls only from 0.464 to 0.425. We also find that embedding task configuration and passage length can each change reported R@10 by a factor of seven, independent of architecture. These results show why low-resource RAG evaluation should report performance by language condition and query type rather than relying on aggregate scores alone. The dataset and evaluation scripts are available at https://huggingface.co/datasets/RaiyanKhaan/AgriTrust-RAG.

0.54

UAE GovTrust Scope Filtering Gains and Routing Failures in Bilingual Public Service Retrieval

cs.IR ·2026-10-23 ·Aleksandar Mihajlovic
similarity 0.66
Abstract · 14 pages, 2 figures, 5 tables. Reproducibility materials included as ancillary files

Public-service queries can mention several locations while only one controls the requested service. We study this distinction in UAE GovTrust, a controlled English-Arabic retrieval diagnostic comprising 22 authored service summaries and 50 scenarios with 110 query variants. Twelve methods separate lexical and dense ranking, rank fusion, scope filtering, a multilingual cross-encoder, and direct service lookup. On 72 matched answerable queries with bilingual evidence, role-filtered reciprocal rank fusion retrieves a relevant first result for 69 queries (95.8%), versus 47 (65.3%) for unfiltered fusion; filtered dense retrieval and direct lookup each achieve 68 (94.4%). With English-only evidence, excluding zero-score lexical candidates raises filtered fusion from 52 to 64 correct first hits. However, the unchanged router selects the correct action and scope for only 2 of 168 post-audit perturbations covering paraphrases, irrelevant facts, negation, and corrections. These perturbations are deliberately authored diagnostics, not held-out evaluation. Fresh-environment execution reproduces all 1,320 original ranking orders and actions. The results show that correct scope metadata can improve ranking while brittle scope inference can remove the evidence needed to answer. The study evaluates retrieval and routing, not generated answers or legal correctness; independent Arabic and domain validation remains necessary.

0.25

Cross-Lingual Information Retrieval in Tetun for Ad-Hoc Search

cs.IR ·2026-10-25 ·Altedio Araújo, Gabriel de Jesus, Sérgio Nunes
top venue similarity 0.65
Abstract · The EPIA 2025 international conference on Artificial Intelligence

Developing information retrieval (IR) systems that enable access across multiple languages is crucial in multilingual contexts. In Timor-Leste, where Tetun, Portuguese, English, and Indonesian are official and working languages, no cross-lingual information retrieval (CLIR) solutions currently exist to support information access across these languages. This study addresses that gap by investigating CLIR approaches tailored to the linguistic landscape of Timor-Leste. Leveraging an existing monolingual Tetun document collection and ad-hoc text retrieval baselines, we explore the feasibility of CLIR for Tetun. Queries were manually translated into Portuguese, English, and Indonesian to create a multilingual query set. These were then automatically translated back into Tetun using Google Translate and several large language models, and used to retrieve documents in Tetun. Results show that Google Translate is the most reliable tool for Tetun CLIR overall, and the Hiemstra LM consistently outperforms BM25 and DFR BM25 in cross-lingual retrieval performance. However, overall effectiveness remains up to 26.95 percentage points lower than that of the monolingual baseline, underscoring the limitations of current translation tools and the challenges of developing an effective CLIR for Tetun. Despite these challenges, this work establishes the first CLIR baseline for Tetun ad-hoc text retrieval, providing a foundation for future research in this under-resourced setting.

0.36

Evidence-Gated Bilingual RAG for Reliable Consumer Banking Search

cs.CL ·2026-10-15 ·Ahmed Mohamed Mahmoud Rayyan, Rana Mahmoud Mohammed, Aya Abdallah Abdelmonem, Mohamed Khaled Goda Othman, Rahatara Ferdousi
similarity 0.62
Abstract

Consumers often describe what they need rather than use the terminology found on a banking website. For example, a natural-language request ("a card for travel") may not share the keywords of the relevant product page, and the problem is harder in bilingual services, where the same intent may be expressed in Arabic or English. This paper presents an evidence-gated bilingual retrieval-augmented generation (RAG) framework for consumer banking search that operates entirely on local infrastructure. The framework combines structured web-content refinement, hybrid dense-lexical retrieval, intent-aware evidence selection, cross-encoder reranking, and a three-outcome Answer / Related Mismatch / No Answer decision mechanism. On 210 URL-labeled Arabic and English queries, the proposed approach improves Top-1 hit rate from 50.48% to 79.05% and MRR@5 from 0.571 to 0.797 over a hybrid baseline. A separate 26-case end-to-end evaluation achieved 69.2% strict success, with 72.7% of generated answers classified as supported by an automated faithfulness judge. These results are preliminary: the retrieval queries are paraphrases of a small reference set, and the end-to-end test is small and uses an automated rather than human judge, so they do not establish human-verified answer correctness or generalizable performance.

0.21

Query-aware routing for Cross-lingual performance gains in Encoders

cs.CL cs.AIcs.IR ·2026-10-02 ·Akshay Jain, Edward Kim
benchmark similarity 0.60
Abstract

Multilingual encoders can exhibit reduced retrieval effectiveness when queries and relevant documents differ in language, despite strong same-language performance. We investigate whether Finnish and Swedish cross-lingual retrieval can improve while preserving an encoder's existing same-language performance and document index. We combine a query-only low-rank adapter, trained against frozen document embeddings, with deterministic routing based on query and index languages. Cross-language queries use the adapter, while same-language queries use the original encoder. SampoTron, our fine-tuned low-rank (LoRA) adapter alongwith the Nemotron-3-Embed-1B model, improves average retrieval quality across six English, Finnish, and Swedish directions from 0.241 to 0.291 in normalized discounted cumulative gain (nDCG) at rank ten, a 20.9% relative gain on a sampled financial benchmark. All six cross-lingual directions improve, and routing preserves the original same-language performance, including two full-corpus Finnish evaluations. The approach enables selective cross-language specialization with reusable document embedding vectors.

0.15

Dhara: Evidence-Oriented Retrieval for Indian Knowledge Systems

cs.IR cs.CL ·2026-11-08 ·Avinash Holenarasipur Nagendra, Anand Kaniyaar Sriranga Prasad, Rahul Vishwakarma, Trupti Gavit, Anish Amol Tipnis +4 more
similarity 0.60
Abstract · 19 pages. Supplementary material (24 pages) is provided as a separate ancillary PDF

Relevant evidence in Indian Knowledge Systems (IKS) collections may be distributed across passages in different parts of a work, including verses, translations, and commentaries. Locating it requires matching natural-language questions and partially remembered verse fragments to source passages. We present Dhara, a retrieval engine combining hybrid search, chapter- and book-level Entity Notes (EN), Fuzzy Verse Retrieval (FVR), and neural reranking. Five comparisons on the 325 questions in Dhara Bench examine retrieval representations against BM25, EN's contribution, reranker quality and computation time, FVR, and Gemini. BGE-M3 single-vector retrieval exceeds BM25 in nDCG@10, Recall@20, and MRR@20 across all categories. Its scores are broadly comparable to ColBERTv2, with mean retrieval times of 0.108 s and 0.962 s under the same hardware configuration and a smaller memory footprint for chunk storage on disk. Adding EN to a direct branch with a matched candidate count improves nDCG@10 in all twelve categories with Qwen and ten with BGE; the combined ranking is capped at the same candidate count. Qwen improves nDCG@10, Precision@5, and MRR@20 over BGE in every category, at approximately 6.72 times the overall estimated serial computation time. FVR increases Verse-category nDCG@10 by approximately 20 percentage points with either reranker. Gemini is evaluated after source mapping and manual review with relevance annotations visible. The Qwen pipeline with EN and FVR has higher Recall@20 than Gemini in ten categories and stronger complete-evidence retrieval in Multi-hop questions, while Gemini retains early-relevance advantages in several categories. The results inform retrieval choices according to evidence requirements and resource budgets.

0.31

The Vocabulary Gap Is an Equity Gap: Register Mismatch in Retrieval Systems for Public-Benefits Access

cs.IR ·2026-08-28 ·Krish Sapru
benchmark similarity 0.60
Abstract · 6 pages, 3 figures, 2 tables

Retrieval-augmented question answering is increasingly used to help people navigate public-benefits eligibility, yet the documents these systems retrieve from are written in agency register while intended users often ask questions in plain, informal, or non-native English. We show that this register mismatch can turn a high-performing retrieval system into an inequitable one. We construct a controlled benchmark of 51 publicly documented federal benefit-eligibility rules and 25 information needs, each phrased in both agency register and plain user register while keeping the gold passage fixed. Across BM25, TF-IDF, and a term-graph retriever, formal-register evaluation is nearly perfect (Recall@5 96-100%), but plain-register retrieval collapses (Recall@5 36-44%). For BM25, Recall@1 falls from 84% to 16% and Recall@5 from 100% to 44%, a 56-point equity gap on identical information needs. We trace the mechanism to a measurable vocabulary gap: formal queries share 0.63 of their content terms with the gold passage, while plain queries share only 0.11, a 5.9x reduction. A deliberately simple, auditable plain-to-formal lexicon bridge recovers much of the failure, lifting plain-query BM25 Recall@5 from 44% to 80%. The contribution is not a new retriever; it is an evaluation protocol, benchmark, mechanistic diagnosis, and transparent mitigation for a high-stakes social-impact failure mode that standard retrieval evaluation hides.

0.28