CS Paper Radar

Rare Disease Classification in Video Capsule Endoscopy

Amit Kumar Patel, Debesh Jha

eess.IV q-bio.TO ·submitted 2026-09-14 13:55 UTC · v1

Abstract

Video capsule endoscopy offers a non-invasive way to examine the small bowel, however, the rare but clinically important findings that appear only in less than 1\% of labelled frames, provide very less signal for detection. As a result, while models can achieve high overall performance, they almost completely miss these rare findings. We condition each frame's decision on evidence from the rest of that patient's video, not on the frame's own posterior alone, supplying information a single frozen frame can never contain. Building upon Expected Gain Arbitration, a per-frame, macro-F1-optimal correction over the confusion matrix, we further compose it with video-disease-presence conditional resulting in our decoding method Hierarchical Conditional-Expected-gain (HCE). We evaluate HCE on two independent VCE datasets, Kvasir-Capsule and GALAR, against a static temporal-smoothing baseline and an optimal-transport (OT) frame-level correction. HCE significantly improves rare-class recall over a temporal-smoothing baseline on GALAR (2.7--3.1x; 95\% CI excludes zero in both scopes) and shows a similar, though not statistically confirmed, gain on Kvasir-Capsule (up to 3.4x). Against OT, HCE still improves rare-class recall in all four evaluation settings (+0.016 to +0.029), and additionally improves macro-F1 and MCC.

Closest papers

Nearest neighbours by meaning across the whole index.

On the Robustness of Temporal Vision-Language Models for Surgical Endoscopy Videos

cs.CV ·2026-08-14 ·Darakshan Rashid, Raza Imam, Ufaq Khan, Muhammad Bilal, Shazad Ashraf +7 more
top venuebenchmark similarity 0.71
Abstract · Accepted to MICCAI 2026

Temporal vision-language models (TVLMs) offer a reusable, prompt-based interface for surgical video understanding, yet, their robustness under clinically realistic acquisition artifacts in endoscopy remains insufficiently characterized. In practice, degradations such as defocus, haze, motion blur, noise, cautery smoke, and packet loss introduce structured distribution shifts which may compromise video-text alignment. We study the robustness of temporal VLMs under such shifts caused by corruptions in clip frames. We introduce Endo-C6, a compact corruption benchmark of six endoscopy-realistic perturbations evaluated at a fixed high severity, and apply it to public Gastrointestinal (GI) endoscopy and laparoscopic cholecystectomy videos. Under a standardized prompt protocol, we benchmark 3 recent surgical TVLM baselines and analyze robustness in both mean and worst-case settings, spanning 294 dataset-level evaluations. Finally, we present RobustEndoCLIP, obtained by few-shot parameter-efficient tuning with VeRA, outperforming existing TVLM baselines. Our findings show that off-the-shelf TVLMs can exhibit severe worst-case collapse under endoscopy-specific corruptions, whereas lightweight few-shot adaptation can substantially improve corrupted performance and robustness without changing the prompt-based interface. We expect Endo-C6 to support standardized robustness reporting and promote more reliable clinical vision-language systems.

0.29

EndoSCOP-V: A Multi-Turn Video Understanding Evaluation Framework for Multimodal Models in Endoscopy Reporting and Clinical Reasoning

cs.CV cs.AI ·2026-11-07 ·Abhishek Tyagi, Tanmay Jain, Vilas Biradar, Krithi Krishna Koduri, Misba Hul Bashir Masoodi +13 more
published similarity 0.70
Abstract · NeurIPS 2026

Endoscopic diagnosis unfolds over native video, where a single positive finding during traversal of the gastrointestinal tract sets off a multi-minute chain of reasoning over location, morphology, severity and intervention. Existing endoscopy multimodal benchmarks test only static images or single-disease videos, which miss the temporal and clinical reasoning that real-world reporting demands. We introduce EndoSCOP-V, a video-grounded multi-turn evaluation framework for structured endoscopy reporting and clinical reasoning. It contains 1,307 real-world endoscopy and colonoscopy videos (63.2 h; 97% 1080p) averaging about 174 s. Its 20,274 question-answer turns are instantiated deterministically from clinician-filled structured endoscopy-reporting EHRs, so no per-case annotation is synthetic or LLM-rephrased. They come from 1,033 clinician-reviewed templates, about 18 per disease, that span a clinically aligned 9-task x 30-sub-task reporting taxonomy over 54 World Endoscopy Organization (WEO) MST 3.0 diseases. 22% of procedures carry co-existing pathologies, up to six per case. A Markovian context-pruned oracle-forcing protocol evaluates the dialogue turn by turn, injecting the gold answer before each subsequent turn so that per-turn reasoning is isolated from cascading dialogue errors. A panel of 15 board-certified endoscopists sets a 78% strict-accuracy baseline on 1,500 questions. Across 14 open-weight LMMs, the best 27B-class model reaches 59.8% strict accuracy, falling to 53.4% once format-trivial disease-rejection items are excluded. On a between-subjects spot-check, the best of 3 frontier proprietary models trails the clinician panel by 23 pp on strict NDR accuracy. Sub-task analysis reveals consistent gaps in quantitative measurement, multi-variable grading and intervention reasoning on long-horizon video. Data: Hugging Face (EndoSCOP-V). Code: GitHub (abhi25t/EndoSCOP-V).

0.26

SurgCBVR: A Multi-Domain Benchmark for Content-Based Surgical Video-to-Video Retrieval

eess.IV cs.AI ·2026-11-17 ·Mohammad Eslami, Mousa Moradi, Negin Ghamsarian, Nazlee Zebardast, Jennifer L Lindsey +4 more
benchmark similarity 0.64
Abstract

Hospitals record thousands of hours of surgical video annually, yet these archives remain difficult to search by content. Surgeons cannot readily retrieve similar operations, complications, or workflow events. Existing surgical computer vision primarily produces labels rather than retrieving relevant videos. We introduce SurgCBVR, the first benchmark for content-based video-to-video retrieval in surgery. It integrates eight public datasets across four domains-ophthalmic, laparoscopic, robotic-assisted, and colonoscopy-covering six procedure types, 679 cases, and 76,489 clips (106 hours). SurgCBVR comprises two tiers: Scene, retrieving the same operative state across cases and sites, and Workflow, retrieving and localizing phase transitions and deviations. Scene includes 74,765 clips spanning 40 classes, three temporal granularities, and three playback speeds; Workflow contains 1,724 transition clips. Fixed clip lengths and frame counts, together with phase-classifier oracle and duration-only baselines, help assess label and timing shortcuts. We evaluate six generic and surgical foundation models, two video retrieval methods, and contrastive heads trained on SurgCBVR, using within-dataset, cross-dataset, and pooled-gallery retrieval restricted to the same procedure type. Frozen representations barely outperform a timing-only baseline, transition retrieval approaches chance, and trained heads lose up to half their accuracy across datasets. These findings expose substantial limitations in current video representations and establish SurgCBVR as a benchmark for advancing surgery-aware retrieval.

0.13

TRUE-Colon: Exposing a Consistent Transfer Asymmetry in Real-Time Polyp Detection

eess.IV cs.CVcs.LG ·2026-08-13 ·Sebastian Doerrich, Andreas Franz Schwab, Francesco Di Salvo, Shyam Nandan Rai, Hanh Huyen My Nguyen +1 more
publishedcode/data similarity 0.64
Abstract · Accepted to EndoLINA @ MICCAI 2026 (The International Workshop on Endoluminal Intervention Navigation and Autonomy)

Computer-aided detection (CADe) systems for colonoscopy promise to reduce clinical miss rates, yet reliable real-world deployment remains elusive. This translational gap stems in part from a structural flaw in model development: the reliance on curated datasets that under-represent the long negative stretches and procedure-related artifacts characteristic of routine examinations. Training and evaluating architectures strictly on these lesion-centric benchmarks creates an illusion of success, since such benchmarks cannot capture clinically crucial metrics. To expose this gap, we establish TRUE-Colon, a standardized benchmarking protocol that measures key deployment characteristics alongside localization accuracy, and evaluate four real-time architectures (Faster R-CNN, YOLOv8, YOLOv11, RT-DETR) across curated benchmarks (SUN, PICCOLO) and 60 unedited, full-length procedures (REAL-Colon). We observe a consistent transfer asymmetry: models trained strictly on curated clips suffer a severe performance collapse when evaluated on full procedures, whereas procedure-trained models substantially improve rejection of non-polyp content on REAL-Colon, and largely retain their accuracy on curated benchmarks. Beyond transferability, we find that the Transformer detector attains the strongest sensitivity and the earliest, most persistent detections, while the convolutional detectors stay competitive at a higher throughput. Together, these results indicate that both training and benchmarking for deployable CADe should shift from curated, lesion-centric clips toward full-procedure data and deployment-relevant operating points. Source code is available at https://github.com/sdoerrich97/true-colon.

0.38

CapsuleMotion: A Lightweight Real-Time Visual Motion Predictor for Capsule Endoscopy

cs.CV ·2026-09-14 ·Oliver Bause, Julia Werner, Oliver Bringmann
published similarity 0.62
Abstract · 4 pages, 3 figures, 2 tables, accepted at the 22nd Annual International Conference on Body Sensor Networks (BSN2026)

Video Capsule Endoscopy (VCE) is a non-invasive medical examination that allows for the observation of the small intestine, which is otherwise difficult to access. A fundamental challenge persists in the form of their limited size in order to still be swallowable. The resulting restricted battery capacity, however, contradicts with the power-intensive nature of image capture and transmission. Therefore, we propose CapsuleMotion, a patient-specific dynamic capsule behavior that utilizes the available energy in a goal-oriented manner to increase the likelihood of a complete screening of the gastrointestinal tract. By investigating and combining metrics from the on-device image compression, CapsuleMotion predicts the motion between two successive frames. The camera's frame rate will be modified in accordance with the predicted magnitude of motion. Furthermore, prior to entering the small intestine, the capsule operates in a low power mode with a significantly reduced frame rate. In this mode, the LocalizationNet is employed to determine the current organ, provided that motion was predicted. The proposed framework is evaluated on the Rhode Island VCE dataset and deployed on an ultra-low power single-core RISC-V demonstrator with an integrated hardware accelerator. CapsuleMotion demonstrated the capability to reduce electric energy consumption by up to 20.66% in comparison with conventional capsules that lack a dynamic frame rate. Additionally, the accuracy of detecting the entry point of the small intestine has been improved.

0.41

Beyond the Dominant Label: Recovering Coexisting Lesion Evidence with Multiple Instance Learning

cs.CV cs.LG ·2026-10-15 ·Lubin Gan, Jing Zhang, Xinming Zhang, Chengpeng Xu, Siying Wu +1 more
similarity 0.61
Abstract · 7 pages, 2 figures

Whole-slide images may contain coexisting lesions, yet weak supervision typically provides only the most severe diagnosis. We formulate this setting as dominant-label multiple instance learning, where the observed label is the maximum-severity element of a latent lesion set. Standard multiclass MIL treats it as a complete one-hot description and therefore suppresses lower-severity evidence whenever it co-occurs with a dominant lesion. We propose DomSet-MIL, a label-aware framework that aligns the annotation mechanism across bag and instance levels. Dominant-Label Censored Likelihood models the reported diagnosis without declaring unreported lesions absent. Severity-Ordered Residual Routing assigns patch evidence from high to low severity and preserves residual support for secondary findings, while top-$q$ pooling links local routing scores to bag prediction. Experiments across natural and controlled settings show that DomSet-MIL achieves state-of-the-art performance in dominant-label prediction while more effectively recovering latent coexisting-lesion evidence. These findings demonstrate that modeling the dominant-label mechanism can improve diagnostic prediction without sacrificing clinically meaningful secondary findings.

0.28

Accuracy Is Not Enough: An Efficiency- and Statistics-Aware Benchmark of ResNet50 and Vision Transformer for Explainable Multi-Class Gastrointestinal Diseases Classification

cs.CV ·2026-10-25 ·Akwasi Asare
benchmarkhot direction similarity 0.61
Abstract

Automated classification of gastrointestinal (GI) findings in endoscopic images could support early diagnosis, but architectures are often chosen on accuracy alone. We present a controlled, reproducible benchmark of ResNet50 and ViT-B/16 for eight-class GI classification on the public Kvasir dataset (4,000 images), using a stratified split (400 test images) and stratified group 5-fold cross-validation over all images. In cross-validation, ResNet50 and ViT-B/16 reached 92.12% and 91.70% accuracy, and a two one-sided tests (TOST) procedure showed equivalence within a +/-3 percentage-point margin (difference +0.43 points, 90% CI -0.33 to +1.18; p < 0.001). The architectures differed mainly in cost: ResNet50 had 3.6 times fewer parameters and 4.3 times fewer operations, 2.5-fold lower single-image CPU latency (69 vs. 177 ms) and 3.8-fold higher GPU throughput. Cross-validated calibration was similar (expected calibration error 0.021 vs. 0.023). An edge-oriented transformer, MobileViT-S, was most accurate on the test split (95.5%) but lost 9.4 points when an acquisition artefact present in some classes (the ScopeGuide inset) was occluded, whereas ResNet50 was not significantly affected. On 500 unseen polyp images with expert masks, both models' Grad-CAM maps focused on the lesion (3.6 times chance). On a de-duplicated external subset of GastroVision (1,998 images), accuracy fell to 78-80% for all three models and differences between them remained small. Accuracy differences between architectures were thus small relative to the effects of shortcut learning and domain shift, whereas efficiency differences were large. An efficient CNN is a defensible default for resource-constrained screening, but no model is deployment-ready without local validation.

0.29

woma: a real-time foundation model and its fine-tuned models for endoscopy

cs.SE cs.CVcs.LG ·2026-09-14 ·Thang Tran, Lan Dang
code/data similarity 0.60
Abstract · 28 pages, 10 figures (8 in the main text, 2 supplementary), 13 tables (8 in the main text, 5 supplementary). Preprint. Models and run record…

woma is a real-time foundation model for gastrointestinal endoscopy: a network trained without labels on about a million endoscopy frames, from which task models are fine-tuned. We contribute a systematic design for production. Requirements and pass marks were fixed before any run, eight candidates screened under pre-registered rules, self-supervised training taken to a stopping rule, then fine-tuning and deployment optimisation, all on one self-contained library, numbat. We also contribute woma itself with two fine-tuned models, every outcome reported met or missed. Our colonoscopy model finds and outlines polyps, names which colon segment is in view, suggests polyp type and grades bowel preparation. Our gastroscopy model names a station out of 22 protocol sites, flags and outlines lesions, and names one of seven findings. Every number was read on data never seen in training, and shipped weights were chosen on that record. In colonoscopy, 96% of polyps in a six-hospital PolypGen set are found at precision >=0.85, and 19 of 19 polyps across fifteen full REAL-Colon videos at 1.6 false alarms per procedure. In gastroscopy, landmark region is named correctly on 92% of frames from unseen patients, and 37 of 39 held-out neoplasia frames are flagged at specificity 0.91. On one workstation GPU every task runs over 1080p video at about 100 frames per second, faster than PyTorch, ONNX Runtime and TensorRT in all four precision regimes tested. TensorRT comes closest: one pass of our foundation model takes it 3 to 27% longer than ours, and we deliver 6 to 31% more frames per second from frame to results. A second build links no vendor library at all -- our own kernels over Vulkan -- so a site deploys two files and needs no toolkit, no cuDNN and no framework; in f32 it beats the CUDA build on the same card.

0.57