CS Paper Radar

The Interviewer's Perspective: Unpacking the Impact of Real-Time AI Interviewing Assistance on Social Dynamics

Zhe Liu, Jiamin Dai, Joanna McGrenere

cs.HC ·submitted 2026-09-24 18:01 UTC · v1

Abstract

Eliciting rich data in semi-structured interviews is cognitively demanding, prompting recent work to explore real-time AI assistance for interviewers. However, introducing AI into the interviewer-interviewee interaction creates a triadic context whose social dynamics remain underexplored. We investigated how interviewers experience AI assistance for probing during semi-structured interviews. To elicit rich participant reflections, we implemented two variants of AI assistance differing in initiation and granularity in a high-fidelity prototype, ProbeAssist. We conducted a qualitative-first comparative structured observation study where 18 participants each completed three simulated interviews: one without AI and two with different AI variants. Findings showed that participants leveraged AI as a supportive tool but resisted it as an assessor or competitor. As they navigated AI's benefits and interaction costs, tensions emerged around agency, ownership, creativity, and interpersonal communication. We propose three implications for AI-assisted human-to-human interaction: managing social pressure, balancing idea alignment with inspiration, and preserving interpersonal presence.

Comment
23 pages, 4 tables, 5 figures, under review for CHI 2027

Closest papers

Nearest neighbours by meaning across the whole index.

InterviewPlayground: A Validated Simulation Environment for Evaluating AI Interviewers

cs.AI cs.CL ·2026-10-08 ·Jonathan Ivey, Aimee Liang, Arthur Y. S. Wang, Madeline Mandell, Ziang Xiao +1 more
similarity 0.73
Abstract · Preprint. 23 pages

Increasingly, AI interviewers are being developed to elicit open-ended responses in applications like market research, public polling, preference elicitation, and social science research. However, evaluating AI interviewers is challenging because they function in extended, multi-turn interactions where they must adapt to participant behaviors. To address this need, we develop InterviewPlayground, a simulation environment for evaluating AI interviewers using simulated study participants whose behaviors are grounded in social theory. Simulated studies in InterviewPlayground produce an InterviewReportCard, which assesses the performance of AI interviewers using a suite of validated measures. To test whether our simulation-based evaluations predict performance with human participants, we conduct 15 real qualitative studies with five AI interviewers, three interview topics, and 450 human participants and compare them to simulated studies in InterviewPlayground. We find that AI interviewer performance in InterviewPlayground predicts performance in human studies with an average Pearson correlation of 0.86 across 12 measures, and the simulated interactions from InterviewPlayground reproduce key findings from behavioral analysis of AI interviewers in the human studies. Together, these findings support the validity of InterviewPlayground in assessing AI interviewer performance and examining potential failure modes. Our work contributes a simulation environment for AI interviewers supported with empirical validation, and more broadly, a roadmap for future work to develop validated, simulation-based evaluations of conversational AI systems.

0.43

The Relational Capacity Framework: A Lens for AI-Mediated Relationality

cs.HC cs.AIcs.CY ·2026-11-06 ·Emily Tseng, Patrick Gage Kelley, Renee Shelby, Ashley M. Walker
similarity 0.64
Abstract

A growing literature examines the impact of engaging with AI chatbots on our relationality: our ability to form relationships with others, maintain them through life's changes, and exit when needed. Currently, substantial work focuses on purpose-built AI companions (e.g. Replika, Character.AI), with less work investigating socioemotional uses of the more widely-used general-purpose chatbots. In this paper, we present a 100-person investigation into how people engage general-purpose AI for socioemotional needs. Participants' AI uses varied, from practicing specific social skills to seeking additional bandwidth and information as they navigated their social contexts. To explain this heterogeneity, we provide 8 states that describe how AI expands or contracts relationality for people with different levels of skills, bandwidth, and information. We then present a unified conceptual model for the role of technology in mediating human relationality, the Relational Capacity Framework, and outline future research designing appropriate roles for AI in human connection.

0.17

Human-AI Collaboration at Scale: Task Criticality, Agency, and Friction across 250,000 Conversations

cs.HC ·2026-08-17 ·Yijia Shao, Dora Zhao, Vishakh Padmakumar, Jennifer Wang, Diyi Yang
similarity 0.63
Abstract · 17 pages, 6 figures

AI is increasingly embedded in professional work, yet how people actually collaborate with it at scale remains poorly understood. Drawing on a corpus of 249,834 real-world conversations, we conduct privacy-preserving analysis by asking three questions: what work people bring to AI, what roles humans retain in completing that work, and where human-AI collaboration breaks down. We find that users bring to AI not only trivial tasks but also consequential, hard-to-reverse work, with high criticality delegation concentrating in advisory and professional domains. As stakes rise, users engage more actively with AI output, and conversation length nearly doubles from ephemeral to high-stake tasks. We then examine human agency, the degree of control and responsibility users retain in completing the task. Human-led collaboration dominates overall, but users construct their level of agency through how they prompt and iterate. Across different levels of agency lies a common tension: does working with AI build human capability, or substitute for human effort? We examine this tension through AI-supported learning and upskilling, finding that active teaching is common (67% of conversations) but its benefits are unevenly realized. We further show that friction arises in roughly half of all conversations but it is often productive, and users often employ active strategies to recover from friction when it does occur. Together, these results map how humans and AI work together at scale and when that collaboration makes people more capable, and when it doesn't.

0.50

Can AI interview? Comparing AI-led to Human-led Chat Interviews

cs.CY cs.CLcs.HC ·2026-10-16 ·Semra Guven, Tobias Gårdhus, Andreas Bjerre-Nielsen, Anna Rogers, Hjalmar Bang Carlsen
code/data similarity 0.63
Abstract · 37 pages, 3 figures, 6 tables. Submitted to Socius. Code and data: https://github.com/ainterviewer/can-ai-interview

Large language model chatbots have been proposed as AI-interviewers for collecting qualitative interview data at scale, potentially bridging social science's longstanding divide between the breadth of surveys and the depth of qualitative interviews. This paper provides the first direct empirical comparison of data quality in AI-led and human-led interviews. In a randomized experiment (N = 41), university students were interviewed via synchronous text chat by either trained non-expert interviewers or a locally hosted AI system; we evaluate exposure, specificity, and relevance. Median interview duration was similar across conditions, but AI-interviewers asked more than twice as many questions per interview and elicited roughly 65 percent more respondent text per interview. Per-question responses were longer with human interviewers, though not after multiple-testing correction; only AI-condition participants skipped questions. Responses were almost uniformly on-topic in both conditions; for specificity we detect no difference, but the design cannot exclude one of up to three quarters of a point. These results were obtained with a small, locally hosted open-weight model (7B parameters), which addresses concerns about data security, GDPR compliance, and scientific reproducibility. Our findings support the viability of AI-interviewing for large-N qualitative data collection.

0.25

"ChatGPT, what am I missing?": Designing AI Workflows around Professional Task Structure to Shape Analytic AI Use

cs.HC ·2026-09-15 ·Zilin Ma, Suzi Jazmati, Marco Chimenton, Yiyang Mei, Jacqueline Lane +2 more
similarity 0.62
Abstract

General-purpose AI lets users choose what support to request, but leaves them to structure the support a professional task requires. We examine how interactive workflows can embed professional task structure without prescribing how users engage with AI. We designed two scaffolded interfaces around the same negotiation scaffold: one presented a completed AI analysis, while the other supported user-directed, incremental development. A four-condition randomized experiment with 800 participants compared these interfaces with no-AI and an AI chat interface. AI-supported conditions improved preparation coverage over unaided work; the scaffolded workflows further improved coverage over chat. Although the scaffolded workflows produced similar coverage, the user-directed workflow elicited a broader repertoire of analytic requests and lower subjective effort. Professional scaffolding therefore depends not only on displayed structure but on how workflows organize users' engagement with it. Effective professional AI must structure how users and AI build analysis together.

0.19

AI-Induced Social Erosion: Using LLMs for Work Tasks Reduces Help-Seeking Intentions Towards Colleagues

cs.HC cs.CY ·2026-11-18 ·Kevin Chow, Sophia Deng, Bowen He, Hanyi Min, Qiaosi Wang +2 more
similarity 0.61
Abstract · 36 pages, 7 figures, 8 tables, preprint

Generative AI is becoming embedded in knowledge work as an intelligent collaborator to assist in completing tasks. However, this readily available assistance risks displacing key interactions through which colleague relationships are built and sustained at work. This paper leverages the concept of help-seeking behaviour to investigate whether and how AI usage shapes colleague relationships. We conducted an online experimental vignette study where 248 knowledge workers evaluated scenarios that systematically varied whether and how AI was used across different work tasks. Using linear mixed-effects models, we found that AI-enabled work reduced help-seeking intentions compared to non-AI work across tasks, and especially for brainstorming. Visible AI usage was associated with greater help-seeking intentions towards colleagues, and individual contributors' colleague relationships were more vulnerable than managers. Our findings identify opportunities for workplace AI tools, practices, and policies to preserve colleague relationships in increasingly AI-integrated workplaces.

0.13

Human-AI Conversational Behaviors Predict Unassisted Task Performance

cs.HC ·2026-10-07 ·Li Siyan, Federico Bianchi, James Zou, Kaitlyn Zhou
similarity 0.61
Abstract

Assistance from AI tools has supported and improved human performance across domains. However, recent research suggests that these immediate benefits may entail future costs, including diminished performance when AI assistance is no longer available. We study how human-AI interaction behaviors correlate with immediate and future unassisted task performance across two game-based problem-solving user studies ($n=139$ and $n=111$), using a \textit{dialogue act} framework adopted from a tutor-student dialogue taxonomy. In our studies, verbalizing thought processes correlates with higher unassisted outcomes, whereas directly requesting solutions correlates negatively. Similar to these participant-side patterns, assistant explanations of the current problem state are associated with better subsequent unassisted performance, whereas directly providing the next action is associated with worse performance. Qualitative and subtype analyses further show that ostensibly similar reasoning turns can elicit different assistance. Our findings suggest that preserving users' cognitive participation in problem-solving may support performance beyond AI-assisted interaction.

0.08

When the Interviewer Is a Bot: Behavior, Breakdowns, and Trust in MLLM-Led Interviews

cs.HC cs.CY ·2026-08-11 ·He Zhang, Kambinachi Chukwuma, ChanMin Kim, John M. Carroll
published similarity 0.61
Abstract · Accepted to ACM HCOMP 2026

Semi-structured interviews are a cornerstone of qualitative research but remain labor-intensive. We report an empirical study of what actually happens when the interviewer is an off-the-shelf real-time multimodal LLM (MLLM). We built InterviewBot, a voice-based interviewing system that wraps a real-time MLLM with a researcher-authored outline, and deployed it not as a novel architecture but as a research instrument for observing default MLLM interviewing behavior. In a practice study (N=15), participants completed a bot-led semi-structured interview and then a human-led reflection session about that experience. We contribute (i) a turn-level behavioral analysis of an MLLM interviewer (N_turns=428) showing that it is acknowledgment-heavy but probe-light (deepening probes account for 4.9% of all turns), and that 28.7% of question-bearing turns pack multiple questions into one turn despite an explicit one-question-at-a-time instruction; (ii) an inductive catalogue of four data-collection breakdowns (information loss, premature termination, latency, and interruption) observed in a deployed rather than simulated system; and (iii) three social dynamics from participants' reflections: disclosure calibration, where reduced social pressure coincided with shallower elaboration; institutional legitimacy, where trust tracked perceived stakes and what delegation to AI signaled about the organizer rather than conversational competence; and conversational grounding, where content-grounded paraphrase, not generic social filler, was what participants read as listening. We conclude with design implications for depth control, transparent handoffs, and non-templated listening mechanisms in human-centered interview automation.

0.22