CS Paper Radar

Large-Scale Qualitative Research with AI: Infrastructure, Management and Operation of the Socioscope Data Pipeline

Saadi Lahlou, Juan Pablo Caicedo, Shriya Sekhsaria, Valentine Fournand, Paulius Yamin, Helga Nowotny

cs.CY ·submitted 2026-08-30 12:25 UTC · v1
code/data

Abstract

The Socioscope project is a pioneering effort in Large-Scale Qualitative Research (LSQR) collecting comparable, open-ended, multimedia field data on hundreds of cases and using AI to make the material analysable at scale. The domain studied is the food system. The entities documented are the organisations that act in it: farms, processors, distributors, retailers, restaurants; and, at meso level, the actors that shape their environment, such as municipalities, government programmes, banks, NGOs and universities. This paper provides the technical reference for how the resulting data Corpus was built and managed to enable AI-augmented analysis. It describes the data pipeline end to end: the systemic sampling frame; the transaction grid used to capture each initiative's relations within the food system; the social contract that rewards participating interviewees, aiming to sustain access; the operational chain from scouting to interviews, including their uploading, transcription, translation, quality control and curation; the provenance rules (originals are immutable, every transformation is logged); and the installation of equipment, personnel and processes, including ethics and GDPR compliance. In its first phase (2023-2026) the pipeline produced 686 documented cases from 31 countries: some 1,430 hours of recordings, about 450,000 speech turns, and 12.6 million words of transcript. We report costs, metrics, lessons learned and limitations, so that other teams can reuse, adapt, and improve the Socioscope methodology.

Comment
80 pages, 6 figures, 3 tables, 7 appendices. The Data Collection Protocol (V8, 55 pages) is available from the authors on request; its table of contents is reproduced in Appendix 7

Closest papers

Nearest neighbours by meaning across the whole index.

Antipatterns in AI-assisted Qualitative Data Analysis: A Catalog of Temptations and Pitfalls for Software Engineering Researchers

cs.SE cs.AI ·2026-08-28 ·Rashina Hoda, Carolyn Seaman, Victoria Gomes, Rodrigo Spinola
similarity 0.60
Abstract · 12 pages

AI-assisted qualitative data analysis (QDA) offers unprecedented opportunities to streamline software engineering (SE) research, yet uncritical use risks compromising analytical rigor and flooding the field with accelerated production of low-quality research. While tactical best practices will naturally evolve over time, SE researchers currently lack strategic guidance to identify and mitigate methodological risks when attempting AI-assisted QDA. Based on our decades of qualitative SE research expertise and experience combined with an understanding of the emerging landscape of AI-assisted QDA, this paper presents a catalog of antipatterns in AI-assisted QDA - a set of assumptions and practices that initially appear advantageous but ultimately undermine analytical rigor and validity. The antipatterns are grouped into three categories reflecting escalating impact: Dangerous Drivers, Operational Missteps, and Analytical Failures. As more SE researchers attempt AI-assisted QDA, these antipatterns will help them identify and avoid common temptations and pitfalls, while reviewers can be equipped with the vocabulary and criteria to call out problematic and failed practice. Ultimately, this catalog of antipatterns can serve as a stepping stone in our responsible methodological evolution toward principled and meaningful human-AI collaboration in qualitative research.

0.38

Moral Missions: Sustaining Responsible Practice by Surfacing Practitioner Moral Decision-Making Procedures

cs.HC ·2026-09-14 ·Teanna Barrett, B. Biira, Jainaba Jawara, Andrew Shaw, Ziwei Dong +6 more
published similarity 0.59
Abstract · To be published in AIES 2026 without appendix

A growing ecosystem of techniques, toolkits, and guidelines has been developed to help data scientists consider the social implications of data-driven technologies. However, prior literature highlights that even when this ecosystem of techniques is provided to professional data scientists, they still struggle to consistently adopt a responsible data science practice. We posit that the key to sustained responsible data science practice is to approach it as a moral mission: a conviction-driven technical practice that seeks to transform social conditions by any degree possible. In this paper, we present a semi-structured interview study with 15 responsible data scientists and AI practitioners to understand the moral decision-making procedures they use to articulate and actualize their moral missions. Through a phenomenological analysis of our participants' accounts, we find participants engage in embodied introspection, circumvent institutional expectations, and center relationality throughout their moral missions. We also present how our participants engage in similar processes to contend with generative AI (GenAI) in their responsible practice. We conclude by calling for subversive data science communities and identifying sociotechnical design implications to better support sustainable responsible data science practice.

0.19

Rethinking Data Quality for AI-Driven Systems: Evidence from Practitioner Interviews

cs.SE cs.AI ·2026-09-25 ·Hariharan Gopinath, Jan Bosch, Helena Holmström Olsson
similarity 0.57
Abstract · This is a preprint version and the final version will appear in the proceedings of PROFES 2026

Data quality research has usually treated data as an input that is stored, processed, and validated. In AI-driven software-intensive systems, data also shapes model behavior, evaluation, and lawful use. Empirical evidence remains limited on how practitioners define, assess, and manage quality under these conditions. We interviewed 16 practitioners from nine organizations and analyzed the transcripts using reflexive thematic analysis and developed six themes from participants' accounts. In AI systems, traceability shifted from modular debugging to attributing model behavior, while using models as quality assessors introduced circularity. Agent context and memory became data objects, and synthetic and pseudo-labeled data made authenticity a quality concern. In foundation-model development, lawfulness became a gate for training data, while representativeness was judged through coverage of situations in which the system must behave safely. Prior ML research examines many of these problems separately. Our study provides a practitioner-grounded account of how they are encountered together as an engineering and organizational concern. We also interpret five recurring conditions as helping explain how the themes relate to reduced trust in data and AI outcomes. We synthesize these findings through lifecycle assurance: a conceptual framing focused on producing evidence that data can support a specific AI claim when its influence may be embedded in model behavior, model-based judgments, or agent actions.

0.28

Toward Agentic AI-Ready Workflows in Built Environment Organizations: Knowledge Structures and Interoperability Challenges

cs.AI ·2026-11-01 ·Ahmet Bahaddin Ersoz, Justin McHenry, Yevgeniya Kovalchuk, Edward Barrett, Mandana Sarey Khanie
published similarity 0.57
Abstract

The rise of agentic artificial intelligence (AI) in the built environment is shifting attention from isolated AI tools toward autonomous systems that reason, plan, and act across project and organizational workflows. The effectiveness of such systems depends not only on large language model capabilities, but also on the readiness of heterogeneous data environments across the Architecture, Engineering, and Construction (AEC) sector. These environments combine structured sources, such as CAD and BIM models, schedules, cost databases, and asset registers, with unstructured ones, including technical reports, design narratives, planning and policy documents, contracts, meeting minutes, and risk registers. This paper reviews how built environment data can be organized to support agentic AI-ready workflows, focusing on knowledge structures, interoperability, and access and integration mechanisms such as retrieval-augmented generation (RAG) and the Model Context Protocol (MCP). Drawing on existing literature, it examines approaches proposed to organize structured and unstructured data and considers how such data can be contextualized and made accessible to AI agents in a reliable manner. The paper synthesizes current perspectives, highlights challenges around fragmentation and interoperability, and identifies open questions for future research on AI-compatible information management in the built environment.

0.31

Data Patching

cs.HC ·2026-09-23 ·ATM Mizanur Rahman, Syed Ishtiaque Ahmed, Sharifa Sultana
similarity 0.56
Abstract

To provide data-driven quality services to their citizens, every institution determines the acceptability of the citizen data and datafied mechanisms through institutional protocols, including preset standards and policies. However, data meeting standards at one institution might fail to meet a different institution's standards in subsequent phases of the intended citizen services due to mismatched protocols, leading to data devaluation in the citizen service ecosystem. Terming this phenomenon cross-institutional data devaluation, we investigate its causes and workarounds through interviews with 41 Bangladeshi participants. We found that traditional data auditing mechanisms cannot solely address data devaluation; hence, we draw on our findings and theory of cross-institutional AI audits to propose the Citizen-centered Cross-institutional Data Audit (CCDA). We also discuss design and policy implications of CCDA in HCI and datafied citizen services.

0.35

Treadstone: A Social-Media-Inspired Platform for Multi-Agent Collaborative Data Analysis

cs.HC ·2026-09-17 ·Hyunwook Lee, Sungbeom Cho, William Benjamin, Changhee Lee, Hyotaek Jeon +4 more
published similarity 0.56
Abstract · 11 pages, 10 figures, 2 tables, 2 page-length appendix. Accepted to VIS 2026 and will be published on IEEE Transactions on Visualization and…

Coordinating human analysts with autonomous AI agents faces the same challenges as human-to-human collaboration: sharing intermediate results, avoiding conflicts, and maintaining group awareness. Current tools rely on unstructured messaging or single-threaded chatbot interaction, which lack the structure to track evolving hypotheses or link claims to evidence. We propose agentic social data analysis, a collaboration paradigm extending social data analysis with a shared coordination feed modeled on the content timeline in social media services. We instantiate this concept in TREADSTONE, a platform where human and AI agents asynchronously post, link, and contest analytical claims via threaded messages within a shared feed. By allowing agents to proactively broadcast hypotheses and enabling users to steer the analysis through lightweight curation, Treadstone seeks to balance machine autonomy with human analytical control. A qualitative user study shows that Treadstone fosters collaboration while preserving human analytical agency, in contrast to the solitary experience of conventional chatbot interaction.

0.56

EJAI: Judgment-Preserving Civic AI for Data Center Environmental Justice

cs.HC cs.CLcs.CY ·2026-11-10 ·Evan Pačes-Wiles, Bryen Alvarez, Sara Gulabani, Akshara Kolipaka, Snehalkumar 'Neil' S. Gaikwad
benchmarkhot direction similarity 0.56
Abstract

Communities in the United States and around the world are contesting data centers, the physical infrastructure of AI. These disputes pose environmental justice issues: residents must weigh unevenly distributed burdens using evidence scattered across metrics, corporate claims, policy documents, and public testimony. A formative study with 10 users showed that dashboards and single-agent AI can settle questions before people examine that evidence. Here we present EJAI, a source-grounded system that links an interactive map, retrieved evidence, missing-data statements, and two stakeholder personas that allow people to inspect evidence and form their own policy recommendations. In a small functional benchmark of 21 queries, retrieved evidence supported 98.4% of factual claims and the system acknowledged missing evidence in 32 of 33 cases. Persona alignment carried a grounding cost: the Corporate Executive omitted some evidence that conflicted with its assigned priorities. In a within-subjects study, 15 participants acting as residents prepared for a simulated town hall. Some checked cited AI claims while treating map metrics as official, some made conditional recommendations with safeguards such as water-use disclosure, and some found the personas simplified or biased. Our results suggest that civic AI can support preparation for public reasoning when it makes evidence, gaps, and disagreement open to inspection, and that corpus imbalance and two-sided personas remain open justice concerns.

0.47

Ontology-supported AI Model and Dataset Management

cs.AI ·2026-08-21 ·Jan Novacek, Ali Ahari, Tobias Müller, Sebastian Reiter, Alexander Viehl +1 more
published similarity 0.56
Abstract · Published in: 2024 IEEE 22nd International Conference on Industrial Informatics (INDIN)

Recently, there has been a great deal of research into improving AI methods and their application. The main focus is on tracking progress, enabling transparent comparisons, and fostering a more profound understanding of AI. In that process, different organizations generate and use plenty of assets that need to be tracked, traced and managed. Moreover, it is important to discover assets relevant for the task at hand. This paper presents research aiming to contribute to answering the question of what is required to exchange and manage AI models and related assets effectively without semantic gaps in an industrial context. We introduce a platform for AI model exchange, which facilitates the usage, exchange, and analysis of AI models and datasets. The platform incorporates an ontology that can foster a more profound common understanding of what is required in these tasks and help tackle the issues mentioned above. Finally, we elucidate the utility of the platform through the illustration of a use case in the context of real-time critical systems.

0.48