CS Paper Radar

OA-MAP: Evidence-Grounded Multi-Agent Multimodal Framework for Interpretable Knee Osteoarthritis Progression

Sixu Chen, Mingrui Yang, Qiang Guan, Xiaojuan Li

cs.AI cs.MA ·submitted 2026-10-08 15:23 UTC · v1

Abstract

Knee osteoarthritis (KOA) progression prediction can support patient monitoring, requiring the integration of multimodal data and multidomain expertise. Moreover, isolated risk estimates provide limited insight underlying a prediction. To automate the progression assessment workflow and reduce manual effort while providing interpretable findings and supporting evidence, we present OA-MAP, an autonomous multi-agent framework for evidence-grounded assessment of structural and pain progression in KOA. The system incorporates modality-specific agents including MRI, X-ray, and clinical agents, together with a coordinator agent. This framework can autonomously recruit specialist agents, select tools for prediction and analysis, and retrieve literature as external evidence based on user request and available patient information. An uncertainty-informed human-in-the-loop mechanism enables clinicians to review and correct intermediate findings, triggering recomputation of affected results. We evaluate the prediction models using 600 participants from the FNIH Osteoarthritis Biomarkers Consortium cohort. On the test set of 100 participants, the fusion models achieve AUROCs of 0.80 for structural progression and 0.68 for pain progression. A case study illustrates how OA-MAP combines risk estimates with intermediate findings, cross-modal conflicts, literature support, and uncertainty indicators to support interactive review.

Closest papers

Nearest neighbours by meaning across the whole index.

CS-LDM: Clinically Guided Structure-Aware Latent Diffusion for Knee Radiographic Forecasting

eess.IV ·2026-10-29 ·Shaotian Li, Chenhui Wang, Wenhua Wu, Yanqiu Wu, Xiaohan Yu
similarity 0.67
Abstract

Knee osteoarthritis (KOA) is a leading cause of physical disability and currently has no curative clinical treatment. Early diagnosis and accurate prognosis are therefore vital for managing its progression. Existing methods mainly predict future severity grades and provide limited visual explanation of structural change. Recent generation-guided approaches synthesize follow-up radiographs for progression assessment, but generated images are often judged by predictive accuracy or image realism, so they may satisfy a grading model while remaining morphologically unreliable. We therefore propose CS-LDM, a clinically guided, structure-aware latent diffusion framework that forecasts patient-specific future knee radiographs from the current X-ray and clinically available current KL grade, using LoRA adaptation, current-grade conditioning, and frozen-teacher diagnostic distillation. Experiments on the public Osteoarthritis Initiative (OAI) dataset show that CS-LDM improves future severity-grade consistency while maintaining competitive image fidelity. Further ordinal, OARSI, ROI-level, and stable/progressed analyses show that radiographic forecasting should be evaluated by balancing diagnostic consistency, anatomical plausibility, and structural preservation.

0.20

FRAC-MAS: A Safe and Explainable Multi-Agent System for Fracture Diagnosis

cs.AI ·2026-08-22 ·Hardik Iyer, Tirath Bhathawala, Mihir Panchal, Ying-Jung Chen, Kiran Bhowmick +2 more
publishedcode/data similarity 0.66
Abstract · Medical Image Computing and Computer Assisted Intervention (MICCAI) 2nd Workshop on Agentic AI for Medicine (AgenticMed)

Fracture detection and its clinical interpretability see notable improvements when deep vision models are integrated with agentic AI architectures. While deep learning models achieve high diagnostic performance, their black-box nature limits clinical adoption. We propose FRAC-MAS, an agentic AI system for automated, explainable, and safe bone fracture detection. The framework combines a stacked ensemble of four vision models with conformal prediction to produce statistically grounded differential diagnoses, while a multi-agent workflow performs independent verification, retrieves clinical guidelines, and generates patient-friendly reports. A pipeline-depth ablation study confirms that our multi-agent critic triages 86.6% of cases into a high-confidence auto-confirmed cohort while escalating uncertain cases, outperforming a single-agent baseline. Patient preference studies against Llama, MedGemma, and Gemini further demonstrate significantly more comprehensible clinical reports. These results suggest that integrating multi-agent critics with conformal guarantees enables safer radiology triage while preserving clinician oversight. More broadly, FRAC-MAS demonstrates how cooperative agentic architectures can serve as auditable, human-in-the-loop decision support systems for safety-critical healthcare. Our code is available at https://github.com/hardik1712/FRAC-MAS, and the website is available at https://frac-mas.vercel.app.

0.41

Knee3DVLM: Dual-Sequence Full-Volume Vision-Language Modeling for Comprehensive Knee MRI Assessment

cs.CV cs.AI ·2026-10-06 ·Maryam Baizhigitova, Andrew Seohwan Yu, Po-Hao Chen, Naveen Subhas, Sixu Chen +5 more
similarity 0.65
Abstract · 11 pages, 2 figures, 5 tables

Vision-language models (VLMs) are increasingly being applied to three-dimensional medical imaging, but their application to knee MRI remains limited, particularly for interpreting the complementary sequences used in clinical practice. We introduce Knee3DVLM, a sequence-aware VLM that uses full-volume DESS and fluid-sensitive TSE MRI to predict 57 anatomically resolved binary diagnostic targets derived from the MRI Osteoarthritis Knee Score (MOAKS) for structured reporting. We evaluated DESS-only, TSE-only, and paired DESS-TSE configurations using subject-disjoint Osteoarthritis Initiative partitions. In a held-out cohort of 1,074 examinations, the fused model achieved 72.98% average accuracy, 71.17% balanced accuracy, 78.96% mean ROC-AUC, and 78.74% macro ROC-AUC, the highest values among the three configurations. In a secondary multiclass analysis aligned with the released 3DReasonKnee cohort, Knee3DVLM was numerically higher than the strongest reported 3DReasonKnee configuration across five pathology categories. These findings support dual-sequence full-volume modeling for comprehensive knee MRI assessment.

0.23

Multimodal Fusion Methods for Whole-Slide Images and Clinical Data in Inflammatory Bowel Disease

cs.CV cs.AIcs.LG ·2026-11-03 ·Ankana Banerjee, Mark van der Giezen, Ole Gunnar Aasprong, Emiel A. M. Janssen, Øyvind Steinsbø +2 more
similarity 0.63
Abstract · 5 pages, 4 figures, Submitted to the 2027 IEEE 24th International Symposium on Biomedical Imaging (ISBI)

Multimodal AI fusion of histology and clinical data is an active research area in computational pathology, but fusion methods are rarely compared under identical conditions and across different clinical endpoints. We present a modular framework in which slide-level embeddings, obtained by training an attention-based aggregator over patch embeddings from a frozen foundation model, and per-visit clinical embeddings are fed into 15 interchangeable fusion methods with a shared output dimension and prediction head (except decision-level methods). Using a prospective cohort of newly diagnosed inflammatory bowel disease patients, we predicted STRIDE-II-aligned remission outcomes from baseline or follow-up visits. The best fusion method reached macro-F1 of 0.57-0.75, outperforming both single-modality models in four of six scenarios, and differed by outcome and input. Image reliance, the drop in balanced accuracy when the slide embedding is removed, tended to follow the remission type - highest for histological and lowest for endoscopic remission; methods with similar accuracy relied on the slide to very different degrees.

0.24

Report-Derived Labels, an Image Teacher and the Gap to Expert Labels in Knee MRI: An Analysis of One Entry to the RSNA 2026 Knee Abnormality Detection Challenge

eess.IV ·2026-10-28 ·Soheil Kooklan
benchmarkcode/data similarity 0.62
Abstract · 24 pages, 10 figures, 14 tables. Code: https://github.com/soheilkooklan/rsna-knee-report-labels

In the RSNA 2026 Knee Abnormality Detection challenge, 58 of 4,407 training studies have expert labels for 12 findings, and the rest have only a radiology report. We compare training targets built from a language-model reading of the reports and from a public image model against two references: the expert labels and the 39,248 study-finding cells that the reports state as present or absent. The report labels and the out-of-fold predictions of the image model reach similar agreement with the expert labels (macro AUC 0.892 and 0.896) but fail on different findings. The report is better for ligament tears and patellofemoral osteoarthritis, which it never calls present when the experts do not; the image model is better for effusion, Baker's cyst, contusion, and synovitis, where the reports call the finding absent, leave it out, or call it present more often than the experts. Their equal mixture reaches 0.927 (gain over the report labels 0.035, 95% interval 0.019 to 0.052), also after removing the studies whose knee side was handled inconsistently. We did not observe a model-level gain from this better target: a 2.5D ResNet50 trained on the report labels alone scored 0.877 on the private leaderboard, after the deadline, against 0.872 for the same model trained on the mixture, and the out-of-fold advantage of the mixture is confounded by a change of code and by possible leakage through the image model's folds. A scanner prevalence prior raised agreement with the reports but not with the experts, so a report-derived reference can reward reporting habits. A rank correction for co-occurring findings gave small out-of-fold gains and no visible leaderboard effect. Our entry, a forked public ensemble with that correction, was displayed at rank 582 of 6,299 on the private leaderboard (0.943, a score shared by 287 teams). All results are exploratory; none of the models is meant for clinical use.

0.41

MedTRACE: Tool-Augmented Multimodal Clinical Reasoning Agents for Evidence-Grounded Decision-Making

cs.CL ·2026-09-13 ·Ji Lu, Lifei Liu, Haoran Yu, Xianglong Wang, Yiru Fang +3 more
published similarity 0.62
Abstract · Accepted to the 22nd International Conference on Advanced Data Mining and Applications (ADMA 2026)

Multimodal clinical decision-making requires reliable reasoning over heterogeneous evidence from electronic health records, medical images, and physiological signals. Existing models typically map these inputs directly to diagnoses without explicitly assessing evidence sufficiency, tool-use requirements, or diagnostic uncertainty. This paper presents MedTRACE, a tool-augmented multimodal clinical reasoning agent for evidence-grounded decision-making. MedTRACE uses modality-specific encoders to construct a unified patient-state representation and performs an iterative loop of hypothesis formation, toolaware deliberation, and evidence verification. It dynamically invokes visual grounding, evidence retrieval, and structured parsing tools to locate diagnosis-relevant regions, retrieve clinical knowledge and similar cases, and extract structured findings. The acquired evidence enters an evidence memory, where a consistency verifier confirms or revises the current hypothesis. MedTRACE outputs a diagnosis together with supporting evidence, an auditable reasoning trace, and calibrated confidence. Experiments on multiple multimodal clinical diagnosis benchmarks show that MedTRACE improves diagnostic accuracy by 5.4% and AUROC by 4.7 percentage points over the strongest baseline. It also improves evidenceselection F1 by 8.2 percentage points and visual-grounding IoU by 6.5 percentage points, reduces expected calibration error by 31.6%, and decreases unsupported diagnostic errors by 27.8%. These results demonstrate that active evidence acquisition and verification improve the accuracy, interpretability, and reliability of multimodal clinical decisionmaking.

0.13

LUCAID: Agentic Multimodal AI for Lung Cancer Precision Pathology

cs.CV cs.AIcs.LG ·2026-08-24 ·Marie-Lisa Eich, Kai Standvoss, Timo Milbich, Alexander Möllers, Miriam Hägele +28 more
similarity 0.62
Abstract

Lung cancer tissue diagnostics is complex, as therapy decisions in precision oncology rely on the integration of histomorphological, immunohistochemical, and molecular features. Yet pathological assessment remains largely visual and semi-quantitative and shows interobserver variability, while existing artificial intelligence (AI) tools cover only selected tasks, rarely reach generalizable expert-level performance, and lack prospective clinical validation. To address these challenges, we developed and clinically validated LUCAID, an agentic AI system for precision lung cancer pathology. An integrative agent couples diagnostic reasoning with nine modules that cover the full routine workflow, from quality control, tumor detection and segmentation, histological subtyping, tumor microenvironment profiling, tumor cellularity quantification, and predictive biomarker scoring (PD-L1, MET, TROP-2) to automated structured report generation. LUCAID enables users to interactively query the module outputs and generate reports that contextualize the results. Against large-scale expert ground-truth annotations, the analysis modules achieved F1 scores of 0.82-0.95. In prospective clinical validation, LUCAID reached 93.0% concordance with an expert-panel adjudicated reference standard across clinically actionable decisions, compared with 68.3-81.1% for five experienced thoracic pathologists.

0.26

Anchor-driven Multi-modal Multi-scale Expert Selection for Survival Prediction

cs.CV ·2026-10-06 ·Tao Zhou, Ying Hu, Huazhu Fu, Yi Zhou, Xiao-Jun Wu +1 more
top venuecode/data similarity 0.61
Abstract · 15 pages, 6 figures, 7 tables

The integrative analysis of histopathological Whole-Slide Images (WSIs) and transcriptomic profiles holds significant promise for cancer survival prediction. However, existing methods typically project multi-modal features directly into a shared latent space without explicit alignment, leading to the entanglement of mismatched morphological cues and molecular signals. Furthermore, current fusion strategies often treat the extreme spatial heterogeneity of WSIs uniformly, lacking mechanisms to adaptively prioritize clinically relevant tissue scales for individual patients. To address these limitations, we propose an Anchor-driven Multi-modal Multi-scale Expert Selection (AM$^2$ES) framework for survival prediction. Specifically, we present an Anchor-driven Multi-modal Fusion (AMF) module, which introduces learnable semantic anchors as cross-modal mediators to bridge the semantic gap by enforcing a structurally regularized alignment between transcriptomic features and multi-scale pathology representations. Built upon this aligned semantic space, we further design a Hierarchical Mixture-of-Experts (H-MoE) selection module to decouple the hierarchical prognostic selection process. Mimicking the pathologist's diagnostic workflow, H-MoE performs (i) Intra-scale Expert Filtering to discriminatively identify salient tumor regions within each magnification, and (ii) Inter-scale Hierarchy Routing to dynamically weight and select the most informative resolution levels. Extensive experiments on multiple TCGA cancer cohorts demonstrate that our AM$^2$ES achieves state-of-the-art performance while offering fine-grained interpretability by visualizing how specific molecular pathways drive the expert routing decisions across tissue scales. The code will be released at https://github.com/taozh2017/AM2ES.

0.47