CS Paper Radar

AraEngage: Interpretable Linguistic Correlates of Engagement in Arabic Facebook News

Shimaa Ibrahim, Nour Elhouda Dalloua, Md. Rafiul Biswas, Wajdi Zaghouani

cs.CL cs.CYcs.SI ·submitted 2026-11-05 12:32 UTC · v1
published

Abstract

Engagement features developed for English are routinely applied to Arabic, but unvocalized Arabic changes what a lexical feature measures. We introduce AraEngage, a query-defined sample of 10,000 public Arabic Facebook news posts from 2,451 pages in 57 countries (2023-2024) that pairs engagement counts with interpretable linguistic features. We use it to examine which interpretable Arabic indicators are associated with follower-normalized engagement, and how far those associations can be trusted given publisher context, the definition of engagement, and lexical ambiguity. Religious expressions, message length, sentence length, and lexical density are the leading correlates in a Random Forest model ($R^2$ = 0.38 on a post-level split) interpreted with SHAP. Unlike English news, positive sentiment is associated with higher engagement in this Arabic news sample. Message length shows a length paradox: unadjusted raw engagement is highest for long posts, whereas the model's attributions for follower-normalized engagement favor short posts. A linguistic audit identifies why standard lexicon features are fragile in unvocalized Arabic: imperative lists contain homographs such as the form read as "express!" that is also the frequent preposition "via", hedge lists conflate the possibility marker qad + imperfect ("may") with the emphatic qad + perfect ("has indeed"), and religious lists mix devotional formulas with place names in political news. These are plausible sources of measurement error that may attenuate the near-zero associations of imperatives and hedges. All findings are observational associations with operationalized indicators. We release the derived linguistic features, engagement counts, refined lexicons, and analysis code; raw posts are not redistributed, in line with CrowdTangle terms.

Comment
13 pages, 4 figures, 8 tables. To appear in ArabicNLP 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Beyond the Keyword: A Human-Annotated Benchmark for Arabic Anxiety-Related Discourse

cs.CL cs.SI ·2026-11-05 ·Shimaa Ibrahim, Mabrouka Bessghaier, Md. Rafiul Biswas, Insaf Elouard, Wajdi Zaghouani
benchmark similarity 0.66
Abstract · 13 pages, 1 figure, 6 tables

Keyword retrieval returns expressions of personal anxiety together with many other uses of anxiety-related language. We present a benchmark of 1,000 Arabic Twitter/X posts retrieved with the keyword qalaq ("anxiety/worry") and annotated for six discourse functions. Two native Arabic-speaking annotators labeled every post independently, reaching 51.1% raw agreement (Cohen's $κ= 0.378$). We then reviewed all 489 disagreements and all 511 agreements under the same guidelines. The review changed 120 of the agreed labels (23.5%), which shows that agreements need auditing as well as disagreements. The final labels form a consensus reference; on a random sample of 200 posts, an additional annotator agreed with it on 60.5% of posts ($κ= 0.517$). Against this reference, Fanar-C-2-27B reaches 79.8% accuracy and macro-F1 0.784 ($κ= 0.749$). Its errors concentrate in the residual category (41.1% of errors; precision 0.905, recall 0.447), whose definition is internally inconsistent in the executed prompt, so the low recall cannot be attributed to the model alone. Sociopolitical Anxiety (27.1%) outnumbers Personal Emotional Expression (22.0%); these are assignment proportions in the sample, not estimates of prevalence or retrieval precision. We release the guidelines, a residual-subtype scheme, a fixed data split, label provenance, and evaluation code.

0.35

AraCrypto-ML: A Multi-Label Corpus of Communicative Functions in Arabic Facebook Cryptocurrency Discourse

cs.CL ·2026-11-11 ·Mabrouka Bessghaier, Mohamed Rashad Rouine, Md. Rafiul Biswas, Youssef Ayadi, Wajdi Zaghouani
similarity 0.66
Abstract

Arabic cryptocurrency discourse on social media is communicatively heterogeneous: a single post may report prices, explain protocols, promote investment, warn of scams, or invoke Islamic jurisprudence. Yet no annotated resource captures this functional diversity. We introduce AraCrypto-ML, a 10,000-post multi-label corpus of Arabic Facebook cryptocurrency discourse (2014-2024) from 2,898 public pages using a set of related keywords. Collected posts are annotated into six categories: Market Commentary, Educational, Promotional, Skeptical-Warning, Religious Framing, and Other/Unclear, using Fanar-2-27B under a prompt-only multi-label protocol. We validate 1,000 posts against adjudicated gold labels (macro kappa = 0.74) and apply Rogan-Gladen calibration to correct labelling bias. The two annotators agree with each other at macro kappa = 0.62 before adjudication, below the model's agreement with either of them (0.70 and 0.69). Reaction profiles differ strongly by function: Love is about 70% of reactions per post on Educational and Promotional posts but about 33% on Skeptical-Warning, which draws the largest Haha share (about 39%).

0.22

Digital diglossia: Arabic between X and Facebook

cs.CL ·2026-09-23 ·Fahad Al Hussen, Mohammed Q. Shormani
unusual similarity 0.63
Abstract · 7 Tables, 3 Figures

This study highlights the distribution of Standard Arabic (SA; H(igh) variety) and Colloquial Arabic (CA; L(ow) variety) across X and Facebook. 16754 public posts were collected via Python, with 10000 retained as the net dataset. Posts were classified into 7 discourse categories: *politics, technology, science, business, culture, fun,* and *sports*. Bivariate analyses, including Chi-square tests and Cramer's V (CV), examined associations among platform, discourse category, and diglossic choice, while binary logistic regression with Platform x Discourse Category interactions tested whether these associations varied across platforms. Findings reveal that there are significant associations between discourse category and diglossic choice on X, chi-square(6, *N* = 5000) = 600.35, p < .001, CV = .347, and Facebook, chi-square(6, N = 5000) = 1249.52, p < .001, CV = .500. Across platforms, platform was also associated with diglossic choice, chi-square(1, N = 10000) = 262.16, p < .001, CV = .162. Binary logistic regression further shows higher odds of SA use on X than Facebook in the political reference category (*OR* = 1.31, p = .0028), with significant platform-by-domain interactions for Culture (OR = 2.65), Fun (*OR* = 6.34), Sports (*OR* = 26.71), Science (OR = 0.41), and Technology (OR = 0.71). The study concludes that the diglossic use of SA and CA contributes to the growing body of research on digital discourse, unveiling that the digital age reshapes but does not erode diglossic boundaries, giving rise instead to a reconfigured digital diglossia.

0.26

Mawqif-XT: An Arabic Benchmark Dataset for Cross-Target Stance Detection

cs.CL ·2026-08-10 ·Rasha Albalawi, Nuha Albadi, Hamzah Luqman, Maram Kurdi, Saad Ezzini +2 more
benchmarkhighly cited similarity 0.62
Abstract

Publicly available Arabic datasets for target-specific stance detection remain limited, particularly for evaluating cross-target generalization. This paper presents the Mawqif-XT, consisting of 996 manually annotated Arabic tweets collected from three public targets: Women Driving, E-Cars, and Trimester System. Each tweet is annotated with stance, sentiment, and sarcasm labels following the original Mawqif annotation scheme. The released extension is intended as a held-out evaluation set for assessing model generalization to both semantically related and previously unseen targets, while the original Mawqif dataset is used for training and development. In addition, we establish baseline results using several Arabic and multilingual transformer models, as well as zero-shot large language models (LLMs), to facilitate reproducible evaluation. Together with the original Mawqif dataset, the Mawqif-v2 Extension provides a benchmark for evaluating cross-target generalization in Arabic stance detection.

0.43

A Comprehensive Analysis of Arabic Natural Language Processing Research: Trends, Topic Evolution, and Research Gaps -- A Bibliometric and Topic-Based Study

cs.CL ·2026-08-24 ·Mullosharaf K. Arabov
similarity 0.61
Abstract · 38 pages, 16 tables, 10 figures. Preprint

Arabic Natural Language Processing (NLP) has grown rapidly over the past decade, driven by digital transformation in the Arab world, social media, and large language models (LLMs). Despite this growth, a comprehensive quantitative meta-analysis remains absent. This study presents a bibliometric and topic-based analysis of 7,120 Arabic NLP papers published between 1960 and 2026, sourced from five platforms (arXiv, ACL Anthology, Semantic Scholar, Crossref, OpenAlex) plus an additional targeted OpenAlex subset. We employ BERTopic for topic modeling, regression analysis, social network analysis, and geographic mapping. Our findings show a significant publication surge after 2020, driven by transformer models and LLMs. Topic modeling identifies 19 themes, the largest centered on text, speech, translation, and recognition (2,942 papers). Citation analysis reveals a positive correlation between paper age and citations (r = 0.245, p < 0.001); regression (R^2 = 0.105) shows that indexing in OpenAlex or Semantic Scholar and institutional affiliation are associated with higher citations. Saudi Arabia, the United States, and Egypt lead in research output. A task-dialect gap matrix identifies understudied areas, including summarization for Maghrebi, Iraqi, and Sudanese dialects. The largest topic has the highest H-index (90), followed by sentiment analysis (57). Our quantitative approach complements existing qualitative surveys and offers recommendations to prioritize under-resourced dialects and develop culturally aligned benchmarks.

0.31

Affective publics in Arabic YouTube

cs.CY ·2026-09-03 ·Lynnette Hui Xian Ng, Craig Douglas Albert, Abdullah Melhem, Ahmed Aleroud, Lance Y. Hunter
similarity 0.61
Abstract

What is the emotional register of Arabic YouTube's affective publics? To investigate this, we analyzed 67,725 YouTube comments collected around socio-political topics associated with Yemen, Saudi Arabia, Iraq, Jordan, and Syria using a unified sentiment-and-emotion pipeline. Our results profile a single regional affective public rather than five separate national ones. Sentiment is overwhelmingly negative across all five country-oriented corpora, and the country-level emotion profiles are structurally similar. This shared register still accommodates some regional variations: discourse is organized around country-level political actors and cross-border historical trauma figures, and grief singularizes Iraq from the other countries. The differences in emotional register also tracks lived political causes rather than fixed categories, which we observe from patterns of the valence of US-related content, that tracks the presence or absence of direct US military engagement. Our work shows that the emotional register of Arabic YouTube's affective publics is a shared one that is historically layered and geographically conditioned, which has implications for public diplomacy in the MENA region.

0.24

AraDynFact: Dynamic Evaluation of Factual Knowledge in Arabic

cs.CL ·2026-09-28 ·Ignacio Iacobacci, Faroq Altam, Zhaozhi Qian, Muhammad Alqurishi
top venue similarity 0.61
Abstract · Accepted to EMNLP 2026 Industry Track

As Large Language Models (LLMs) continue to scale both in size and capabilities, their proficiency in the Arabic Language has seen significant advancement. However, a critical gap remains: the extent of their factual knowledge and cultural sensitivity to the diverse Arabic-speaking world remains largely underexplored. Current evaluation metrics often focus on translation or generic reasoning, failing to capture the rich historical, social, and regional nuances inherent to Arabic culture. In addition, most benchmarks rely on heavy work, with human intervention in some steps, making the evaluation of knowledge coverage expensive and slow. To address this deficiency, we introduce AraDynFact, a novel dynamic evaluation framework designed to rigorously assess the factual Arabic knowledge embedded in LLMs. Unlike static benchmarks, AraDynFact employs a dynamic approach to extract factual information and generate rich and answerable questions in a fast and automatic way. We apply AraDynFact to Arabic Wikipedia and audit the performance of several state-of-the-art models, ranging from Arabic-centric specialized LLMs to high-resource general purpose LLMs. In addition we found a high degree of correlation with existing, hand-crafted Arabic-centric benchmarks, confirming the potential of our dynamic approach.

0.26

Beyond Fluency: A Rubric-Based Benchmark for Evaluating Saudi Dialect and Cultural Competence in Large Language Models

cs.CL cs.AIcs.CV ·2026-08-30 ·Ghassan Al-Sumaidaee, Sajjad Abdoli, Ahmed Rashad, Maxim Legg
benchmark similarity 0.60
Abstract

Large language models are increasingly deployed in Arabic-speaking markets, yet standard benchmarks overwhelmingly reward Modern Standard Arabic (MSA) fluency while leaving dialectal and culturally grounded competence unmeasured. This gap is consequential: everyday Arabic is largely dialectal, and dialect encodes social meaning that MSA-centric evaluation cannot capture. We present a rubric-based benchmark for the Saudi dialect, comprising 31 expert-authored prompts spanning idiomatic, pragmatic, lexical, and culturally-embedded phenomena, each paired with an expert-established ground truth. Our methodology separates evaluation into a model-agnostic phase, in which atomic, MECE positive criteria are derived solely from the ground truth, and a model-specific phase, in which four state-of-the-art systems -- Claude Opus 5, Gemini 3.7, GPT-5.6, and Kimi K3 -- are scored against those criteria and penalised for errors they actively introduce. Across 124 model-prompt evaluations we catalogue 466 error instances under a nine-category taxonomy. The four systems cluster within a narrow macro-average band (42.7%-53.1%), with no model exceeding 55% and every model recording at least one negative-scoring prompt, confirming that Saudi dialectal competence remains broadly unsolved. Notably, Ambiguous Framing is the dominant failure mode (37.3% of errors) while outright Hallucination accounts for only 11.2%, indicating that models fail less by stating falsehoods than by distorting register and flattening pragmatic nuance. We further observe a consistency-versus-ceiling trade-off and model-distinctive error signatures. We release the full prompt set, ground truths, and scored rubrics to support reproducible dialectal evaluation.

0.24