CS Paper Radar

Layer-Wise Gate-Controlled Prompt Truncation in a Multimodal Chest X-Ray Classifier

Jingtao Lei, Hongji Li, Dexiang Shu

cs.LG cs.AIcs.CV ·submitted 2026-09-06 13:15 UTC · v1
code/data

Abstract

Mixture of Prompt Experts (MoPE) adapts multimodal transformers through input-dependent prompt composition, while retaining a fixed prompt length. We investigate a layer-wise gating extension in a binary chest X-ray classification pilot study. The controller predicts a retention ratio for each sample, averages these ratios within a mini-batch, and uses the resulting integer length to truncate the static and mixed visual prompts. Retained mixed prompts are also scaled by the individual ratios. In one recorded run per configuration, the gated model reached a best validation accuracy of 0.8996, compared with 0.8969 for the fixed-length baseline; the corresponding final values were 0.8963 and 0.8802. The exported gate statistics imply a retained length of one at all recorded training points, relative to a configured maximum of six. This reduces the complete visual sequence from 210 to 200 tokens, but no direct runtime measurements establish an acceleration benefit. Report-derived labels, report text as input, sequential data partitioning, and the absence of repeated controlled experiments limit interpretation. The findings document prompt shortening under the configured gate penalty; they do not establish sample-specific length allocation, superiority over fixed short prompts, or clinical utility. Code is available at: https://github.com/jingtaolei/mope-dynamic-prompt-truncation.

Comment
12 pages, 3 figures, 3 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Physiologically Informed Digital Auscultation for Pneumonia Detection in Long-term Care Residents

cs.SD cs.CVcs.LGeess.AS+1 ·2026-09-23 ·Nicholas Rasmussen, Oleg Zaslavsky, Zih-Ling Wang, Hongyu Yu, Joelle Fathi +3 more
similarity 0.61
Abstract · Manuscript has been submitted to NPJ Digital Medicine

Pneumonia is difficult to diagnose in older long-term care residents; multimorbidity and atypical presentations obscure signs, motivating operationally efficient objective testing. We analyzed multi-channel digital stethoscope recordings from 185 Japanese residents (73 pneumonia, 112 symptomatic without), using radiologist-confirmed chest X-rays and clinician diagnoses as supervisory signals that train convolutional neural networks, multimodal fusion, and channel-based variants with time-domain Grad-CAM interpretability. Models were evaluated with repeated patient-level cross-validation showing models with X-ray supervision outperformed clinician supervision (F1 0.729, accuracy 0.783 vs. F1 0.637, accuracy 0.711). Additionally, a three-channel selection protocol maintained performance (F1 0.736; accuracy 0.803), with two mid-thoracic sites ranking highest and Grad-CAM attention overlapping adventitious sounds. These findings indicate automated multi-channel lung-sound analysis can aid long-term care pneumonia diagnosis, with X-ray supervision being more reliable than clinical, and fewer channels preserving performance while lowering acquisition times.

0.14

ICGF-CXRMate: Image-Conditioned Clinical Context Compression and Gated Fusion for Chest X-ray Report Generation

cs.CV ·2026-10-22 ·Vu Minh Tran, Thuc Huynh, Son Thanh Luu, Doanh C. Bui
similarity 0.60
Abstract

Auxiliary patient data can improve chest X-ray report generation, but directly concatenating heterogeneous clinical information with visual tokens produces long, variable-length decoder prompts and may expose the model to redundant or image-irrelevant context. We propose ICGF-CXRMate, an image-conditioned clinical context compression and gated fusion framework. Learnable queries are first conditioned on the current chest X-ray representations and then attend to the patient-data sequence, compressing it into a fixed number of clinical context tokens. Gated cross-attention selectively integrates these tokens into the visual stream, while the compressed tokens are retained in the decoder prompt as a direct clinical pathway. Experiments on linked MIMIC-CXR and MIMIC-IV-ED examinations show that ICGF-CXRMate improves report quality while reducing the average prompt length. In Stage~2, it improves RadGraph-F1 from 24.00 to 28.86, CXR-BERT similarity from 56.86 to 62.80, and BLEU-4 from 5.14 to 7.96 compared with CXRMate-ED, while reducing the average prompt length from 373.89 to 288.37 tokens, corresponding to a 22.9\% reduction. After self-critical sequence training, ICGF-CXRMate outperforms CXRMate-ED on six of eight report-quality metrics in Stage~3. Ablation studies confirm the complementary contributions of image-conditioned compression, gated fusion, and direct retention of the compressed clinical tokens.

0.17

Trading Parameters for Accuracy in MambaVision for Chest Radiograph Classification

cs.CV ·2026-11-15 ·Qaiser Farooq, Tehseen Zia
similarity 0.59
Abstract · 10 pages, 4 figures, 9 tables. Submitted to an IEEE journal

MambaVision, a hybrid state-space and attention backbone, places most of its parameters in its widest, fourth stage. We quantified what that stage contributes to multi-label chest radiograph classification. The fourth stage was replaced with lighter single-scan state-space blocks adapted from Spatial-Mamba, with multi-scale dilated depthwise convolutions applied to the scan output; the first three stages were retained. Tiny, Small, and Base models were evaluated across nine dataset-scale combinations drawn from two source cohorts: the National Institutes of Health ChestX-ray14 archive (full ChestMNIST and a high-resolution subset) and Indiana University Open-i. At the Tiny scale, replacement reduced parameter count by 35.7% and increased graphics processing unit throughput by 19.3%; the savings came from the stage's dense projections, not from removing quadratic attention. Without distillation, the mean area under the receiver operating characteristic curve on ChestMNIST was 0.004 lower than the baseline (95% bootstrap confidence interval, 0.002 to 0.006), and 0.003 to 0.006 lower across all nine combinations. Knowledge distillation from an unreplaced teacher narrowed the gap to the undistilled baseline to 0.001-0.002 and met the equivalence criterion on ChestMNIST, but distilling the baseline with the same recipe improved it by 0.001-0.003, so a gap of 0.002-0.005 persisted under symmetric training. At the Base scale, the unreplaced Small model was both smaller and more accurate. Fourth-stage replacement is a useful trade-off only for Tiny and Small models. These measurements let developers of chest radiograph screening tools weigh model size and speed against a quantified loss of diagnostic discrimination.

0.27

Beyond Accuracy: Quantifying Pulmonary Attribution in Anatomy-Guided Chest X-Ray Classification Under Domain Shift

cs.CV ·2026-08-31 ·Abdullah Al Mamun, Md. Nasif Osman Khansur, Md Ashraful Hossen Akash, Md. Kishor Morol, Tze Hui Liew
code/data similarity 0.59
Abstract · 34 pages, 8 figures, 7 tables. Code available at https://github.com/Abdullah-229/BeyondAccuracy

Deep-learning models can achieve strong chest X-ray (CXR) classification performance without establishing whether their predictions predominantly rely on pulmonary image content. This study evaluates pulmonary attribution containment as an anatomy-related reliability property distinct from diagnostic performance. We propose DBCA-SegNet-MGAP, a multi-task anatomy-guided CNN-Transformer framework that combines complementary feature representations through bidirectional cross-backbone attention, predicts a soft lung mask, and incorporates this anatomical prior directly into classification through Mask-Guided Adaptive Global Average Pooling (MGAP). Pulmonary attribution containment is quantified using the Anatomical Local Energy Ratio (ALR) and high-intensity cumulative ALR (cALR@0.9). Experiments were repeated across three training seeds using the COVID-19 Radiography Database for four-class internal testing and a locked Shenzhen-to-Montgomery protocol for zero-shot external tuberculosis testing. On COVID-19, the proposed model achieved a weighted F1 of $0.9615 \pm 0.0015$ and macro ROC-AUC of $0.9906 \pm 0.0007$. In an architecture-matched dual-bridge comparison, replacing conventional GAP with MGAP increased ALR from $0.3878 \pm 0.0098$ to $0.7086 \pm 0.0104$ and cALR@0.9 from $0.5265 \pm 0.0101$ to $0.9905 \pm 0.0018$, while weighted F1 remained essentially unchanged ($0.9618 \pm 0.0015$ vs. $0.9615 \pm 0.0015$). Under locked external transfer to Montgomery, ROC-AUC remained $0.9080 \pm 0.0043$ and pulmonary ALR remained $0.6466 \pm 0.0081$, whereas weighted F1 decreased to $0.7528 \pm 0.0080$ and ECE increased to $0.1683 \pm 0.0055$. These findings show that diagnostic discrimination, calibration, and pulmonary attribution containment are distinct model properties and support their joint evaluation under internal testing and external domain shift.

0.51

Med-AR: Autoregressive Vision-Language Pretraining for Long-Tailed Chest X-Ray Classification and Uncertainty-Aware Evaluation

cs.CV cs.AI ·2026-09-24 ·Janhavi Prabhu, Sahil, Akshay V, Shivam Shukla, Manoj Tadepalli +1 more
similarity 0.58
Abstract · 80 pages including supplementary material, 28 figures, and 22 tables. Supplementary material is included

Long-tailed chest X-ray classification requires visual representations that capture both common abnormalities and subtle, infrequent findings. We propose Med-AR-8B and Med-AR-2B, two radiology-native autoregressive vision-language models pretrained with structured reports, abnormality-focused text, and region annotations. We evaluate the transfer of their visual encoders to multi-label classification against contrastive, self-supervised, and supervised pretrained encoders, including Med-CLIP, CheXFound, EVA-Base, ARK, and BioViL-T, using a common ML-Decoder classification head. To assess fine-grained recognition, we also construct LLM-expanded, report-derived label sets for MIMIC-CXR and CheXpert. Across PadChest, MIMIC-CXR, and CheXpert, Med-AR-8B outperforms Med-CLIP in mean AUROC and AUPRC for head, medium, and tail findings. On MIMIC-CXR, it increases tail-label mean AUPRC from 0.1033 to 0.1441. Med-AR-2B achieves the strongest discrimination results on PadChest. Across the broader encoder comparison, a Med-AR variant achieves the highest mean AUROC and AUPRC in every reported prevalence group on each public dataset. Both Med-AR variants also achieve lower excess area under the risk-coverage curve than Med-CLIP on all three public datasets, indicating improved selective-prediction performance under the evaluated protocol. Internal results are metric-dependent, with Med-CLIP retaining advantages in overall and tail AUPRC and in selective prediction. These findings establish Med-AR as a strong pretraining recipe for long-tailed chest X-ray classification on the evaluated public benchmarks and demonstrate the value of assessing discrimination and selective prediction together.

0.28

Multimodal Error Detection in Chest X-ray Reports: A Large Language Model Reading Image Evidence

cs.CL ·2026-11-19 ·Yanda Cheng, Bo Wen, Yuhao Lu
similarity 0.58
Abstract · 34 pages, 4 figures, including supplementary material

Rad-Linter checks chest X-ray reports against image-derived evidence to flag potential errors for review. A pretrained image classifier supplies scores for nine pathologies, and a 1.5B-parameter language model, fine-tuned with LoRA on 1,816 examples, checks their consistency with the report. We compare it with a keyword-and-negation rule baseline given the same reports and scores. Evaluation uses controlled report-score pairings from OpenI, with consistency labels derived from automatic report labels and score thresholds. Each counterfactual quad crosses two real reports with two real score vectors from other patients, creating two consistent and two inconsistent pairs. Every report and score vector occurs in both conditions, so either input alone permits only 50% accuracy. On 131 quads, Rad-Linter achieves 98.5% pair accuracy versus 80.5% for the rule baseline, and classifies all four pairs correctly in 94.4% of quads versus 56.5%. The checkers perform similarly on omission items, including unmentioned and negated findings; larger differences occur on the remaining item types, which require interpreting how findings are described. The same language model without fine-tuning remains near 50% accuracy. On a second set of 181 quads that also includes reports on which the automatic labelers disagree, pair accuracy is 93.4% versus 75.3%. These controlled results show that, given the same image-derived scores, a trained language model detects report-score inconsistencies more accurately than the tested rule baseline, with the advantage concentrated in interpreting how findings are worded.

0.28

GazeRIB: Systematic Evaluation of Multi-Reader Gaze-Guided Chest X-ray Learning

cs.CV cs.HCeess.IVeess.SP ·2026-10-30 ·Nattakorn Kittisut, Roy M. Gabriel, Marly van Assen, Mohammadreza Zandehshahvar, Carlo N. De Cecco +2 more
similarity 0.58
Abstract · 20 pages on main manuscripts (3 tables + 4 figures), 14 pages supplementary (10 tables + 8 figures)

Radiologists' visual attention can supervise medical image learning, but how reader panel composition affects its value remains untested. We evaluated GazeRIB (Gaze-guided Radiologist-Informed Blackout), which masks unviewed regions during training and uses unmasked images at inference. Eight experts repeatedly read the same 80 chest X-rays (CXRs). We compared individual readers with three-, five- and eight-reader panels across seven architectures and 10 seeds, evaluating 910 models on a separate 2,897-CXR cohort for four-class severity prediction. Every panel improved macro F1 over the no-gaze baseline by 5.33-7.47 percentage points (pp; Bonferroni-adjusted p < 0.007); no individual reader reached significance on this endpoint. Three readers captured 73-80% of the eight-reader gain in macro F1, the area under the receiver-operating-characteristic curve (ROC-AUC) and the area under the precision-recall curve (AU-PRC). On external validation without retraining, all four panels improved ROC-AUC on SIIM-FISABIO-RSNA by +1.93 to +4.24 pp (p <= 0.004); on MIMIC-Eye no panel differed significantly from baseline, whereas the single-reader average declined (-1.40 pp, p = 0.008). Reader composition therefore influenced performance, although transfer depended on the cohort.

0.17

Look What the Probes Dragged In! Real-World Chest X-ray Shortcuts in MedCLIP

cs.CV cs.LG ·2026-08-12 ·Nikolette Pedersen, Regitze Sydendal, Veronika Cheplygina, Théo Sourget
publishedcode/data similarity 0.57
Abstract · 11 pages, 3 figure, poster presentation at the joint FAIMI, BRIDGE, and EPIMI workshop at MICCAI 2026 (Strasbourg, France) conference

Vision-language models, such as contrastive language-image pre-training (CLIP)-based approaches, have reached state-of-the-art (SOTA) results in medical artificial intelligence. However, recent work reveals that CLIP-based models remain vulnerable to shortcuts. We investigate how real-world shortcuts manifest across different layers of the medical CLIP-based model, MedCLIP, and its vision encoder, a frozen ResNet-50. We attach 17 linear classification probes to the intermediate layers of the ResNet-50 and train them on three different dataset configurations and targets: NIH-CXR14 (pneumothorax) and PadChest (cardiomegaly and pneumothorax). This setup allows us to observe model behaviour during evaluation using subgroup-based calibration and layer-wise confidence curves. We find that the final linear probes achieve a high AUROC but poor calibration in the models. The layer-wise confidence analyses suggest that shortcuts emerge at different depths. Patterns consistent with localised shortcuts, such as drains, appear at later layers, while patterns consistent with diffuse shortcuts, such as scanner-specific noise patterns, emerge earlier, aligning with previous work. Finally, we conduct a manual analysis of the images, which reveals data quality issues in both NIH-CXR14 and PadChest. Our findings underscore that even SOTA models remain vulnerable to shortcuts, and the need for high-quality and well-annotated datasets to draw solid conclusions. Code can be found on our GitHub: https://github.com/nikodice4/MedCLIP_shortcuts.

0.43