CS Paper Radar

Towards Trustworthy Multi-stakeholder Recommender Systems

Katarina Marcincinova, Adrian Gavornik, Matus Mesarcik, Michal Kompan

cs.IR ·submitted 2026-10-30 12:41 UTC · v1
publishedunusual

Abstract

Recommender system (RS) nowadays has to reflect, combine, and solve often contrasting requirements and expectations from dozens of stakeholders. In fact, the problem is even more complicated as each of the stakeholders can simultaneously have various objectives creating a diverse and complicated environment with several stakeholders and their objectives in which a RS operates. To address this problems the Multi-objective and Multi-stakeholder RS have been studied in the literature. However, a limited attention has been paid to trustworthiness aspects of such RS, which slowly becomes a new standard. In this paper, we highlight open challenges and important questions which need to be addressed on the way to trustworthy Multi-stakeholder RS.

Comment
https://proceedings.mlr.press/v294/marcinc-inova25a.html
Journal ref
Fourth European Workshop on Algorithmic Fairness, MLR 294:446-451, 2025

Closest papers

Nearest neighbours by meaning across the whole index.

RepuLink: A Linked Data Platform for Accountable Trust

cs.SI ·2026-08-22 ·Wenbo Wu, George Konstantinidis
similarity 0.49
Abstract · ISWC 2026: 25th International Semantic Web Conference, Poster and Demos

Trust and reputation systems underpin reliable interactions in large, distributed networks. However, conventional models typically propagate trust only forward, offering no accountability for endorsers regarding whom they vouch for, and leaving newly joined nodes without a meaningful initial reputation. RepuLink addresses these limitations by proposing a two-layer trust and reputation model that integrates direct interaction feedback with domain-specific endorsements. Crucially, it holds endorsers accountable via Backward Endorsement Penalty/Reward Propagation (BEPP/BERP). This paper demonstrates RepuLink-Tool, a deployable, full-stack reference implementation of this model. The application enables nodes to interact, rate, and endorse each other, while tracking reputation via a live dashboard and an interactive trust network graph. Furthermore, we introduce a new Linked Data layer built on top of the application. This layer features a lightweight OWL ontology encompassing nodes, interactions, ratings, endorsements, pairwise trust assessments, and computed reputation scores annotated with PROV-O provenance. It also provides an on-the-fly RDF projection of each user's trust network in multiple serialisations, alongside a scoped SPARQL endpoint that nodes can query live against their own data.

0.27

A Code-Level Audit of X

cs.CY cs.SE ·2026-11-12 ·Luka Bekavac, Simon Mayer
similarity 0.49
Abstract · Submitted. Currently under peer-review

The transparency regime of recommender systems on Very Large Online Platforms (VLOPs) is undergoing significant change under the EU Digital Services Act (DSA). The DSA requires platforms to disclose information about their recommendation pipelines and grants vetted researchers access to non-public data, which may extend to recommender code. However, it is not known to what extent such code releases enable substantive and meaningful third-party auditing of a production recommender system. We address this question by proposing a code-level audit methodology for recommender systems and applying it to the X platform, which has voluntarily open-sourced its recommender pipeline. Our methodology is a backwards-walk traversal of the repository, tracing each decision-relevant component from the user-facing timeline back to its upstream producers, checking that it is reachable from the served pipeline, and classifying each under a four-tier inspectability scheme. Applying this procedure, we identify 13 transparency dead-ends on the live serving path: models, services, label lists, and configuration values that the public code depends on but the release does not contain. We find that the components that decide which posts are filtered and how they are ranked fall into the two least inspectable tiers, a pattern that intensifies with the 2025 integration of the proprietary Grok language model into moderation, annotation, and ranking. We further show that the release keeps 2023 code that the 2025 pipeline no longer reaches, with 36 parameter references that would pass for dead-ends without a reachability check. Finally, we discuss which additional artifacts would be required for meaningful auditing.

0.18

You Know What You Will Like: People Predict Their Own Ratings Better Than Recommender Systems

cs.HC econ.GN ·2026-11-04 ·Zihan Chen, Lei Nico Zheng, Di Zhu
similarity 0.49
Abstract · Working paper. 23 pages, 3 figures, 10 tables

Recommender systems usually learn what people want from what they have done. Platforms could also ask people directly, but asking is costly: 98.8% of the prompts in our data went unanswered. Theories of statistical prediction imply that a person's own forecast adds little to a recommender algorithm built from many people's behavior. Theories of dispersed knowledge imply that people hold information about their own taste that the data lack. We test the two with the MovieLens Beliefs dataset. In 2023-2024, a homepage prompt asked members how they expected to rate movies they said they had not seen. Of 26,020 such expectations, 551 from 277 members were followed by a rating at least a day later. We compare each expectation with four benchmarks: the platform's own predicted rating, two collaborative-filtering models (matrix factorization and an item neighborhood), and a stack trained on other members' outcomes. The trained models use each member's latest ratings. People predict their own ratings more accurately than all four benchmarks (correlation 0.69, against 0.34 for MovieLens and 0.54 to 0.59 for the trained recommenders). Added to all the machine forecasts and features, the expectation lowers out-of-sample mean squared error by 26%. The gain holds when models are trained on earlier expectations and tested on later ones. A member's earlier expectations about other movies do not replace the current one. In ordering a member's own movies, the trained recommenders are statistically indistinguishable from the person. On old, heavily rated movies, the machines do as well or better. The results reconcile the two theories: the crowd's data suffice where they are rich, and the person adds information about the specific item. We derive four design principles for preference elicitation and recommender evaluation.

0.26

Beyond Raw Engagement: A Counterfactual Observability Framework for Recommender Systems at Netflix

cs.IR cs.AI ·2026-09-19 ·Chaoran Guo, Ding Tong, Ting-Po Lee, Scarlet Chen
similarity 0.48
Abstract

Understanding the performance of large-scale recommender systems remains an underexplored challenge, especially for content creators and model developers. The raw engagement signals available to them, such as views and clicks, conflate content quality, model behavior, presentation bias, and audience reach, making it hard to attribute outcomes to the right cause. In this work, we present a general evaluation framework that enhances observability across multiple recommender systems at Netflix and demonstrate its effectiveness through several production deployments. The framework treats recommender-system observability as a counterfactual measurement problem: estimating what the recommender would have done, and what engagement would have followed, in the absence of a specific content item or model decision. We articulate three stakeholder-centered observability principles for content creators and model developers, and propose measurement methodologies covering bias reduction, relativity, and incrementality, applicable to both single-stage and cascading recommender systems and serving both audiences from a single measurement foundation.

0.21

Multi-criteria Rating and Review based Recommendation Model

cs.IR ·2026-11-01 ·Emrul Hasan, Chen Ding, Alfredo Cuzzocrea
publishedpositionhighly cited similarity 0.48
Abstract · 10 pages. Accepted version of the paper published in the 2022 IEEE International Conference on Big Data (Big Data). (c) 2022 IEEE

These days, due to the advancement of information technology, recommendation system has become one of the key tools for e-commerce business. E-commerce platforms allow users to provide feedback in both written comments and numerical ratings. Recommendation systems are utilized to recommend users to new or unseen items based on these previously collected comments or ratings. In recent years, multi-aspect or multi-criteria based recommendation systems have been studied a lot by the recommendation research community. However, these research works are conducted either with reviews or ratings, not with both. In this project, we argue that integrating both textual reviews (with multiple aspects) and numerical multi-criteria ratings can further enhance the overall rating prediction accuracy. We propose a Multi-criteria Rating and Review based Recommendation model (MRRRec). We show that incorporating multi-criteria ratings into multi-aspect ratings from reviews has a great impact on performance. Our proposed model outperforms several state-of-the-art models such as ANR, DeepCoNN, and Deep Multi-criteria Recommendation System in terms of MSE, MAE, precision, recall, and F1. We show that our proposed model achieves an average of 19% and 23.0% lower MSE and MAE respectively and 7.0%, 1.0% and 3.8% higher precision, recall, and F1 score respectively. We further show that our model performs significantly better with Word2Vec word embedding than the GloVe word embedding method.

0.68

AgentRecommender: LLM Agents Enable Customizable Recommender Systems on the User Side

cs.IR cs.AIcs.DBcs.DL ·2026-09-25 ·Ryoma Sato
similarity 0.47
Abstract

Recommender systems have traditionally been developed for platforms. However, this has given rise to many phenomena that may be advantageous for platform lock-in but are a nuisance to users, such as clickbait, filter bubbles, and the spread of fake news. Recently, user-side recommender systems have been proposed as a new paradigm for solving this problem. If users deploy their own recommender systems, they are no longer at the mercy of the platform's interests. However, building a user-side recommender system is not trivial; in particular, customizing one for oneself requires additional data. We propose AgentRecommender, a method that leverages the investigation capability and internal knowledge of LLM agents to flexibly build user-side recommender systems without additional data. AgentRecommender allows users to easily create recommender systems tailored to their own preferences.

0.19

TrustFormer: Cross-Temporal and Cross- Dimensional Transformer for Task-Specific Multi-Dimensional Trust Evaluation

eess.SY cs.LG ·2026-08-25 ·Botao Zhu, Xianbin Wang
published similarity 0.47
Abstract

In dynamic collaborative systems, the selection of reliable collaborators is critical to ensuring effective task execution. Existing trust evaluation methods often rely on unidimensional or scalar representations, which fail to faithfully capture a collaborator's true trustworthiness, thereby motivating a shift toward multi-dimensional trust modeling. However, due to the asynchrony of collected trust-related data across different dimensions, as well as the complex intra- and inter-dimensional dependencies embedded within these data, multi-dimensional trust evaluation remains challenging. To address these challenges, we propose TrustFormer, a task-specific multi-dimensional trust evaluation framework. Specifically, TrustFormer leverages task identifiers and device-generated timestamps to synchronize heterogeneous trust-related data across historical collaborations. It further employs cross-temporal and cross-dimensional attention mechanisms to jointly model temporal dynamics and inter-dimensional correlations, thereby effectively learning the multi-dimensional trust evolution of potential collaborators from historical performance data. In addition, according to the multi-dimensional resource requirements of tasks, potential collaborators' multi-dimensional resource trust is evaluated. Finally, by synthesizing these multi-dimensional trust profiles, the framework enables the optimal collaborator selection. Experimental results demonstrate that TrustFormer outperforms existing methods by yielding a 40.8% improvement in trust evaluation accuracy and enabling more reliable collaborator selection.

0.38

Multicriteria Recommendation System by Leveraging Predefined, Implicit, and Undefined Criteria

cs.IR ·2026-11-01 ·Emrul Hasan, Chen Ding
published similarity 0.47
Abstract · 9 pages. Preprint (submitted version, prior to peer review). The Version of Record is published in Social Networks Analysis and Mining (ASON…

Data sparsity, a long-standing problem in the recommendation system, occurs when the user-item utility matrix is sparse. For years, many approaches have been experimented with, including utilizing user's textual reviews and multiple criteria ratings. In a review-based recommendation system, user preferences and item attributes are learned from reviews, while in multi-criteria recommendation systems, criteria ratings are used for the recommendation task. Multi-criteria are domain-specific pre-defined explicit criteria and users are bound to provide feedback only on these criteria with discrete values. On the contrary, users are free to express their opinions through textual reviews, which can cover multiple perspectives or aspects. It is possible that these pre-defined, domain-specific explicit criteria are not enough to fully capture the multi-faceted user preferences. To this end, we propose a novel multi-criteria recommendation model that utilizes predefined, implicit, and undefined criteria. We use a semantic similarity-based sentence clustering method to identify the predefined and implicit criteria and a sentiment analyzer to estimate their ratings. The criteria that are extracted from the review and are aligned with predefined criteria are referred to as implicit criteria. A sentence is considered as expressing opinions on an undefined criterion if the similarity score between this sentence and all the predefined criteria is lower than a predefined threshold. Ratings are computed for each extracted implicit criterion and the undefined criterion based on the review content. Finally, we use all three types of criteria and an aggregation model to make the final rating prediction for the recommendation system. Our proposed method demonstrates the superiority compared to several baselines on TripAdvisor and Beer Advocate datasets.

0.49