CS Paper Radar

Candidate Generation and Definition-Guided Verification for Sentence-Level Depression Symptom Recognition

Weiming Li, Catarina Barata, Miguel Constante, Joao Sanches

cs.CL ·submitted 2026-09-01 20:08 UTC · v1
publishedhot direction

Abstract

Sentence-level recognition of depression symptoms is challenging because similar expressions can differ in symptom relevance, and language-model inference is insufficiently grounded in diagnostic definitions. This study proposes a two-stage framework separating symptom-candidate generation from definition-grounded verification. A contrastively fine-tuned sentence encoder generates a symptom candidate per sentence, and a fine-tuned language model verifies whether the candidate is present or absent using the sentence, its context, and a candidate-specific diagnostic definition, checking its judgment against that definition before answering. Evaluated against encoder, inference-based, medical, and general LLM baselines and a matched single-stage supervised classifier, the proposed pipeline attains the best accuracy and F1 scores of all methods, with rationales matching expert-authored annotations. A preliminary clinical audit indicates moderate alignment with diagnostic definitions, with explanation quality strongly dependent on prediction correctness. The results support decomposing symptom recognition into candidate generation and definition-grounded verification, though performance remains limited for rare categories.

Closest papers

Nearest neighbours by meaning across the whole index.

EviDial: When Correct Depression Predictions Lack Evidence

cs.CL ·2026-10-20 ·Junjie Xu, Jin Zhu, Yuzhe Yang, Luhan Wang, Xingjiao Wu +4 more
benchmarkhot direction similarity 0.69
Abstract

Depression dialogue benchmarks commonly assess models by comparing predicted symptom labels with clinical scale scores. However, a prediction may agree with a benchmark label even when the dialogue lacks enough evidence to justify it. We introduce EviDial, a benchmark that distinguishes label agreement from evidence-grounded assessment. EviDial comprises 382 depression dialogues from three sources and 2,744 participant-symptom cells, each classified as having absent, partial, conflicting, or sufficient evidence. Because evidence sufficiency lacks a directly observable gold standard, we derive calibrated evidence references from multiple judgments using generalizability theory, many-facet Rasch measurement, and random-effects latent class analysis. Our audit reveals that 82.8% of participant-symptom cells lack sufficient textual evidence for the corresponding symptom assessment, with substantial variation across sources. Across model families and prompting strategies, 69.2%-96.9% of predictions judged correct by benchmark labels are nonetheless unsupported by sufficient dialogue evidence. Further analyses show that strong agreement among LLM evaluators need not reflect independent evidence judgments, underscoring the need to calibrate automated reference panels. These findings demonstrate that label-based evaluation can overestimate evidence-grounded performance by conflating agreement with clinical labels and support in the observed dialogue. EviDial provides a measurement layer for separating these constructs and auditing LLM-based depression assessment more faithfully.

0.28

Interpretable Symptom Vectors for Depression in a Large Language Model

cs.CL cs.AIcs.LGq-bio.NC ·2026-09-01 ·Fangyi Zhu, Ajay Subramanian, Allison Constant, Camille Wang, Ravish Gupta +1 more
similarity 0.69
Abstract · 26 pages, 6 figures

Patients with depression present with diverse symptom profiles, yet clinical practice routinely reduces this variation to a single severity score. Large language models (LLMs) can potentially capture various symptoms and their severity from patient speech. However, how depressive symptoms are represented inside LLMs remains poorly understood, limiting clinical trust. To examine whether internal model activations match clinician judgment, we analyzed the residual stream of Gemma-3-27B-PT using mechanistic interpretability techniques. Recording activations across symptom descriptions drawn from validated clinical instruments, we found that symptom groups geometrically separated the most at layer 21 across multiple distance metrics. Using Semantic Projection, we then projected held-out naturalistic text onto Symptom Vectors constructed from these instruments. The resulting per-symptom coefficients preserved clinician-annotated rank ordering across mood, somatic, and suicidality axes. Furthermore, a single depression vector in Layer 21 separates held-out depressive from non-depressive text (AUC = 0.789), which can be used as an emotional valence gate that restricts symptom projection to depressive speech. These results reveal a decorrelated, clinician-aligned symptom signal readable directly from internal activations, offering a mechanistic foundation for interpretable depression-assessment tools.

0.58

SIGMA: speaker-isolated, symptom-structured multimodal estimation of depression severity from clinical interviews

cs.SD cs.AIcs.LG ·2026-11-17 ·Zhuoyuan Tang, Tao Wang, Kai Yang, Li Yuan, Yuezhou Zhang +6 more
similarity 0.66
Abstract

Clinical interviews are central to depression assessment but time-consuming and rater-dependent. Automated severity estimation offers scalable screening, yet existing methods often retain interviewer speech (confounding participant cues), fuse modalities late (losing fine-grained cross-modal dynamics), or score symptoms independently (ignoring inter-symptom correlations). We present SIGMA, a speaker-aware, symptom-structured framework for depression-severity estimation from semi-clinical interviews. SIGMA uses speaker diarization to restrict input to participant speech, cross-attention so that text, audio and video condition one another during encoding, and a graded-response head that predicts the eight questionnaire items jointly through a shared latent severity trait with item-specific ordered thresholds. On the Extended Distress Analysis Interview Corpus, SIGMA achieves state-of-the-art performance (CCC 0.762, RMSE 4.13, MAE 3.30 over five seeds). Ablations confirm that isolating participant speech and joint, structured symptom prediction each substantially improve concordance and estimation accuracy.

0.12

A corpus-only AUC diagnostic for pooled speech-based depression detection

eess.AS ·2026-10-22 ·Cezary Polak, Milosz Dudek, Daria Hemmerling
similarity 0.65
Abstract

When clinical speech corpora are pooled for evaluation, corpus membership can itself become predictive if class prevalences differ across sources. We derive the AUC of an oracle predictor that uses only corpus identity and empirical prevalence, a diagnostic computable directly from corpus composition. Across five depression corpora in three languages (590 speakers), the corpus-only AUC is 0.697, and corpus identity is predicted from six acoustic and self-supervised representations with 97.5-99.8% accuracy. Permuting depression labels within each corpus removes label structure while preserving prevalences; the residual pooled-minus-LOCO discrepancy then increases across three nested configurations in the same order as the diagnostic. Pooled 5-fold cross-validation also outscores corpus-held-out evaluation across all six representations. Prevalence equalisation substantially reduces the discrepancy for eGeMAPS and WavLM-Base, whereas a sample-size-matched control does not. Corpus-specific base rates therefore create a measurable shortcut opportunity in pooled clinical-speech evaluation, motivating corpus-aware diagnostics alongside pooled performance.

0.18

SUP-MIMIC: A Multi-Task Clinical Diagnosis Benchmark for Evaluating LLMs' Robustness to Contradictory Evidence

cs.CL cs.AI ·2026-08-30 ·Yi Yu, Bo Wang, Chong Feng, Ge Shi, Xia Liu +2 more
benchmark similarity 0.64
Abstract · 18 pages, 13 figures, 3 table

Current evaluations of large language models (LLMs) primarily focus on factual knowledge retrieval, overlooking the fundamental challenge of navigating the complex, non-bijective mappings between clinical indicators and diagnoses. Existing benchmarks fail to assess whether large language models truly possess the reasoning capability required for diagnostic ambiguity scenarios, where identical clinical presentations may correspond to different etiologies, and diagnostic convergence scenarios, where heterogeneous symptoms ultimately indicate the same disease. To address this issue, we propose SUP-MIMIC, a multi-task framework utilizing MIMIC-IV-v3.1 that comprises Basic Assessment (BA), Diagnostic Divergence Task (DDT), and Diagnostic Convergence Task (DCT). Specifically, DDT is designed to evaluate the model's "one-to-many" disambiguation capability among phenotypically similar cases, while DCT assesses the model's ability to identify "many-to-one" diagnostic patterns across different pathophysiological pathways. Comprehensive evaluation of state-of-the-art LLMs reveals substantial performance degradation on DDT and DCT compared to baseline tasks, exposing a systemic reliance on statistical shortcuts over genuine causal reasoning. Our findings further highlight a conservative bias toward "healthy" predictions, implying non-trivial risks for missed diagnoses in realistic medical settings. This work establishes a rigorous methodology for quantifying clinical reasoning robustness and provides a roadmap for enhancing the safety of language models in clinical medicine.

0.20

From Relevant Posts to Attributable Spans: Explicit Symptom Evidence for Depression Questionnaire Inference

cs.CL ·2026-10-14 ·Sumanth Manduru, Carlotta Domeniconi
similarity 0.63
Abstract

Longitudinal social-media histories offer an opportunity to assess depression, potentially supporting the inference of standardized depression questionnaire responses. However, this task requires evidence that is semantically relevant to a symptom and attributable to the user, while being sufficiently specific to support an individual questionnaire item. Existing approaches rely on semantic similarity to retrieve posts, which can introduce misleading or non-attributable evidence, particularly in multi-symptom posts and discussions of others' experiences. We propose SREE (Symptom Retrieval and Evidence-Grounded InfErence), an evidence-grounded pipeline that extracts first-person and symptom-specific text spans, selects a representative subset using a Facility Location-inspired objective, and performs Evidence-Gated Inference over the selected evidence. We evaluate evidence retrieval on BDI-Sen and evidence-conditioned questionnaire inference on eRisk 2019-2021. Compared with state-of-the-art retrieval approaches, SREE achieves relative improvements of 70% in post-level precision and 59% in F1, while retaining at least 30% fewer posts. On multi-symptom sentences, it also outperforms dense retrieval in precision and F1 even with twice the retrieval budget. Across all three eRisk collections, SREE achieves the highest exact item-level agreement while remaining competitive on ordinal and aggregate metrics. These results highlight the importance of localizing attributable evidence and controlling its composition before inferring depression questionnaire responses.

0.19

Words Are the Voice of the Mind: Depression Severity Estimation from Social Traces via Mental-Guided Evidence Reasoning

cs.CL ·2026-10-24 ·Nada Xu, Xueqin Chen, Qiang Gao, Hongru Liang, Wenqiang Lei
similarity 0.62
Abstract

Depression severity estimation is crucial for mental health assessment and intervention, yet traditional approaches depend on clinical interviews or self-report questionnaires that require users' active participation. With the widespread use of online social media, users leave rich social traces that may reflect their mental states, motivating the task of social media-based depression severity estimation. Existing efforts have progressed from textual-feature-based classification to LLM-based questionnaire completion over standardized scale items, yet they still fall short in user-specific mental priors modeling, diversified semantic retrieval, and fine-grained reasoning over mental evidence. To address these limitations, we propose UMer, a user-specific mental evidence reasoning framework for social traces-based depression severity estimation. UMer first distills a user-specific mental prior from historical social traces to capture the user's psychological tendencies, behavioral patterns, and self-expression style. It then performs scale-guided multi-query retrieval to construct evidence memories from item-relevant posts. Finally, it conducts coarse-to-fine evidence-aware reasoning to distinguish direct and indirect evidence and predict item-level scores with interpretable rationales. Extensive experiments on real-world benchmarks demonstrate that UMer not only achieves competitive performance over existing baselines, but also provides evidence-grounded interpretability and generalizes to other scale-based tasks.

0.23

Augmenting Interviewer Judgments of Patient Experience with Automatic Language Analysis

cs.HC cs.CL ·2026-08-31 ·Aowen Shi, Michal Balazia, Danilo Postin, René Hurlemann, Jan Alexandersson +2 more
published similarity 0.60
Abstract · Accepted to ACII 2026 as an oral presentation

Understanding how psychiatric patients subjectively experienced a clinical conversation is important for feedback and alliance-related process monitoring. While interviewers form post-session judgments about patient experience, these judgments do not always match patients' self-reports. Automatic approaches for predicting perceived interaction quality from conversation have been proposed, but it remains unclear whether such approaches can complement human judgment rather than simply replicate it. To address this gap, we evaluate a clinician-support framework in which post-session interviewer ratings are combined with automatic language-based predictions to estimate patient-reported interaction quality in free clinical interviews. We assess this integration across multiple standard model types, including Ridge, SVR, MLP, GRU, and BiLSTM, all trained on sentence embeddings extracted from dyadic transcripts of 107 free conversations between psychiatric patients and interviewers. Our results show that combining interviewer judgments with model predictions through simple averaging yields the strongest overall performance. The interviewer-only baseline reached a Pearson correlation of 0.365. Among fully automatic models, Ridge achieved the strongest Pearson correlation (r = 0.286), while BiLSTM achieved r = 0.270. The strongest result was obtained by BiLSTM interviewer integration (r = 0.403). Our findings suggest that automatic language analysis and interviewer judgment capture complementary aspects of patient experience and that their combination provides a more accurate approximation of the patient's own report than either source alone.

0.34