CS Paper Radar

Language Distances are Practical for Equitable Cross-Lingual Transfer

York Hay Ng, Razan Ahsan Rifandi, Aditya Khan, En-Shiun Annie Lee

cs.CL ·submitted 2026-09-26 07:49 UTC · v1

Abstract

Cross-lingual transfer is strongly conditional on how the source language is chosen, but it is impractical to determine the best candidate source for every target language, especially for low-resource target languages. Language distances are widely used to rank candidate sources due to their correlation with transfer efficacy and applicability in resource-sparse settings. However, the reliability of distance-based rankers across tasks and resource levels remains underexplored. We therefore present the first equity-focused evaluation of paradigms for ranking source languages, studying resource-level inequality and task inequality across ten cross-lingual tasks and two multilingual models. While both inequalities are most pronounced for individual language distances and an English-always baseline, they are substantially reduced by training-free composite distances, and nearly eliminated by trained rankers. We further demonstrate the reliability of rankers using language distances compared to rankers using language model internals. Overall, we find that language distances provide a practical basis for equitable and performant transfer language selection. We recommend using trained rankers when task-specific transfer evaluations are available, and composite distances otherwise.

Comment
Accepted to MRL 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Dataset Scarcity Limits Robust Evaluation of Multilingual Embedding Models: A Case Study of Slavic Languages

cs.CL ·2026-08-25 ·Ana Gjorgjevikj, Barbara Koroušić Seljak, Tome Eftimov
benchmark similarity 0.75
Abstract

Multilingual text embedding models enable cross-lingual transfer of knowledge across a wide range of NLP tasks, but their evaluation remains highly uneven across high-, mid- and low-resource languages. In this paper, we propose a two-dimensional framework, specifically tailored for analyzing multilingual embedding benchmarks under dataset scarcity, and apply it on the Slavic-language subset of the MTEB benchmark. The framework distinguishes between task-specific and cross-task evaluation, while jointly analyzing three complementary aspects: (1) ranking robustness, (2) model consistency, and (3) evidence strength. At the task-specific level, we evaluate the stability of model rankings under changes in ranking methodology and benchmark dataset composition. At the cross-task level, we assess the ability of models to generalize across diverse tasks within a language. To quantify the reliability of benchmark conclusions, we introduce an Evidence Strength Score that accounts for dataset availability, diversity, and robustness assessability. Our analysis reveals severe benchmark sparsity, with many Slavic language-task pairs relying on a single dataset or highly correlated benchmark collections, limiting the ability to draw robust conclusions. The cross-task analysis reveals a small group of highly transferable models, most notably llama-embed-nemotron-8b, multilingual-e5-large-instruct, and Qwen3-Embedding variants, that consistently perform well across Slavic languages and tasks. Overall, the results demonstrate that benchmark rankings and robustness conclusions must be interpreted jointly with certain notation of their evidence strength and highlight benchmark scarcity as a major obstacle to trustworthy multilingual evaluation.

0.23

DonorRank: Donor Language Selection for Low-Resource Cross-Lingual Speech Recognition

cs.CL ·2026-08-11 ·Akriti Dhasmana, Aarohi Srivastava, David Chiang
similarity 0.72
Abstract · 11 pages, 4 figures, 12 tables

Low-resource automatic speech recognition (ASR) often relies on cross-lingual transfer from higher-resource languages due to limited availability of transcribed speech. However, selecting an effective donor language is a nontrivial issue, particularly in settings with spontaneous speech, linguistic variation, and uneven resource availability. We present DonorRank, a learning-to-rank framework that uses linguistic and dataset-level features along with previously observed transfer outcomes to predict promising donor languages for zero-shot ASR. We evaluate DonorRank on two multilingual speech corpora of Indic and African languages. DonorRank produces informative donor rankings across both collections, outperforming common heuristics in the more closely related Indic setting while remaining competitive in the more linguistically diverse African setting. Our analysis reveals that effective donors are not always the closest linguistic relatives and that the most informative predictors vary across language collections. These findings highlight the value of learned donor selection for prioritizing transfer experiments in low-resource ASR. We will also release all of our evaluation results, feature matrices for all languages, scripts for training the DonorRank framework, and pairwise transfer performance matrices for our datasets.

0.10

Language Chain in Alignment: Cross-lingual Ranking Preference Optimization

cs.CL cs.AI ·2026-08-24 ·Seungyoon Lee, Minhyuk Kim, Jungseob Lee, Heuiseok Lim
top venuecode/data similarity 0.71
Abstract · EMNLP 2026 Main

The alignment of Large Language Models heavily relies on English-centric high-quality preference data, which often leads to suboptimal performance in other languages. In this paper, we propose Cross-lingual Ranking Preference Optimization~(CRPO), a novel framework that leverages robust preference knowledge from English to facilitate preference alignment in the target language. We design a hierarchical structure within parallel preference pairs across the target language and English to jointly optimize intra- and inter-lingual preferences, thereby enhancing language adaptation and output quality. Building on the LambdaLoss framework, CRPO goes beyond the binary comparison based optimization by providing a relative ranking signal across multiple candidate responses. Our experiments across five languages with varying resource scales demonstrate that CRPO consistently outperforms standard approaches in both instruction-following and knowledge utilization capability. Notably, the robust performance gains observed across various weighting schemes further validate the empirical effectiveness of our hierarchical design in a multilingual setup. Furthermore, our findings highlight that CRPO significantly improves both reward margins and the log-probability of desirable responses, contributing to a more stable preference manifold for cross-lingual alignment. Our code is available at https://github.com/dltmddbs100/CRPO.

0.59

Cross-Lingual Representation Alignment by Token-Level Optimal Transport in a Language-Agnostic Space

cs.CL ·2026-09-06 ·Taisei Yamamoto, Ryoma Kumon, Danushka Bollegala, Hitomi Yanaka
top venuecode/data similarity 0.71
Abstract · Accepted to EMNLP 2026 main

Cross-lingual alignment (CLA) aims to align the representations of large language models (LLMs) across languages, enabling cross-lingual transfer to improve multilingual capabilities. Previous CLA methods often ignore language-specific information encoded in representations and only consider sentence-level alignment, which may lead to suboptimal performance and input-output language mismatch. We propose CAROT (Cross-Lingual Alignment of Representations in a Language-Agnostic Space via Optimal Transport), which consists of two steps: identifying language-specific representations in LLMs' internal states and aligning language-agnostic representations across languages at the token level by optimal transport, while explicitly preserving language-specific representations. Inference-time steering experiments show that the representations computed by CAROT are effective alignment targets, improving multilingual performance by up to 11.2 points in accuracy while maintaining input-output language consistency. We further use the representations obtained by CAROT as training targets, internalizing the aligned representations. The trained models outperform existing CLA methods in 11 of 18 evaluation settings (3 models $\times$ 3 tasks $\times$ ID/OOD languages). Our work provides insights into what constitutes effective alignment targets for CLA in LLMs. Code is available at https://github.com/ynklab/CAROT

0.45

Cross-lingual Calibration of Pre-Generation Success Probes for Multilingual LLM Routing

cs.CL cs.AI ·2026-10-05 ·Andrea Paganelli, Stefano Civelli, Pietro Bernardelle, Gianluca Demartini
similarity 0.70
Abstract

Pre-generation success probes estimate response correctness from a language model's hidden activations before decoding, enabling cost-aware routing. While prior work has demonstrated their utility primarily on English inputs, we study their reliability across languages along three dimensions: (1) whether they preserve the ranking of likely successes and failures (DISCRIMINATION); (2) whether they retain probabilities that match observed success frequencies (CALIBRATION); and (3) whether they produce scores comparable enough across candidate models for cost-aware multilingual routing (UTILITY). Using 3,000 MATH problems in 10 languages and 8 open-weight model configurations, we compare cross-lingual transfer from English-trained probes and equal-budget pooled multilingual probes. English-trained probes retain useful cross-lingual discrimination but become less well calibrated after transfer. Pooled multilingual supervision improves both properties and yields more reliable estimates of success. In routing experiments, the pooled router achieves a 0.7% higher test success rate while reducing modeled cost by 13.0% relative to always selecting the model with the highest average success. These results show that multilingual routing requires success estimates that remain well calibrated and comparable across languages and models.

0.09

MoE--LoRA Methods for Multilingual Adaptation: A Systematic Empirical Study

cs.CL cs.LG ·2026-11-17 ·Nikit Srivastava, Joel Dag, René Speck, Axel-Cyrille Ngonga Ngomo
published similarity 0.70
Abstract · To Appear - EACL 2027 (Findings)

Routed low-rank adapters offer a parameter-efficient means to improve the performance of large language models (LLMs) across tasks. Hence, it is assumed that they can also be used to improve the support of low-resource languages in LLMs, which currently serve English far better than the thousands of other languages spoken worldwide. However, this assumption has never been verified empirically under a shared multilingual evaluation protocol. We address exactly this gap by comparing eight low-rank adaptation (LoRA) methods on a frozen base model, using 146 training runs across 200 unique languages and more than 10,000 evaluations. Our results suggest that a reduced-scale search can identify viable configurations of LoRA methods but does not reliably predict their relative performance, as the top-ranking configuration is never best at scale. LoRA reliably improves language-model fit on translation data, consistently degrades reading comprehension, and benefits low-resource languages most. These findings suggest that LoRA methods should be regarded as targeted tools rather than general-purpose means for improving the multilingual capabilities of LLMs.

0.27

Improving Cross-Lingual Transfer for Sequential Sentence Classification in Research Papers via Structural Similarity

cs.CL cs.DL ·2026-09-17 ·Kazuhiro Yamauchi, Marie Katsurai
published similarity 0.69
Abstract · Accepted at JCDL 2026 (ACM/IEEE Joint Conference on Digital Libraries), Frisco, TX, USA, October 13-16, 2026. 12 pages, 5 figures, 9 tables.…

Sequential sentence classification (SSC) is an essential task for structuring scientific publications, and extending SSC research to languages other than English can improve accessibility to scientific knowledge in multilingual digital libraries. Cross-lingual transfer is a promising approach to address the scarcity of training data in non-English languages. Prior work on other natural language processing tasks has shown the benefits of capturing linguistic similarity between source and target languages. However, SSC inherently depends on patterns at the discourse level, such as label sequences and positional regularities, which appear consistently across languages regardless of linguistic differences. To examine the factors that determine transfer success in SSC, we constructed a multilingual SSC dataset covering 13 non-English languages collected from five academic databases. Our cross-lingual transfer experiments, using both encoder-based and generative models, show that linguistic proximity has no consistent predictive power for transfer performance, whereas structural similarity in rhetorical organization shows a weak but consistent positive correlation across models. After controlling for source-language performance, the similarity of label distributions is the most consistent predictor. Building on this finding, we propose a set of three methods that explicitly leverage structural information using generative models. In the in-domain evaluation, the best combination reaches parity with the strongest encoder baselines, and in transfer to languages unseen during training, it outperforms the strongest encoder baseline.

0.25

Apples to Apples? Towards Comparable Crosslingual Language Model Evaluation

cs.CL ·2026-08-25 ·Xiulin Yang, Ethan Gotlieb Wilcox, Catherine Arnett
top venue similarity 0.68
Abstract · EMNLP 2026 CR

Crosslingual evaluation of language models that enables fair comparisons remains a fundamental challenge in multilingual NLP. Existing studies adopt a variety of downstream tasks and intrinsic metrics with different theoretical justifications, yet there has been little empirical investigation into whether these approaches yield meaningful crosslingual conclusions. We systematically examine crosslingual evaluation approaches using controlled monolingual language models trained on parallel data with varying tokenizer vocabulary sizes and model sizes, and further validate our findings on multilingual LLMs. We further discuss challenges in achieving comparable downstream evaluation across languages. Our results show that several widely used normalized metrics introduce crosslinguistic biases rooted in tokenization, encoding, and orthographic differences. In contrast, sentence-level negative log-likelihood computed over semantically equivalent sequences provides more meaningful and consistent crosslingual comparisons.

0.63