CS Paper Radar

Instance-preserving graded semantic reporting for surgical action triplets

Yanchong Xue, Wanting Zhang, Lingtao Yu, Guochao Zhang

cs.CV ·submitted 2026-11-09 01:39 UTC · v1

Abstract

Complete-or-reject reporting of surgical action triplets can discard a correctly identified instrument and action when the target is uncertain. We introduce instance-preserving graded semantic reporting, which separates instance selection from the choice of semantic detail. Each selected instance retains its predicted mask and labels while hierarchical correctness scores guide instrument (I), instrument-verb (IV), or complete instrument-verb-target (IVT) reporting. The utility-increment rule DELTA weighs the expected gain from finer reporting against its error cost. We compare three decision rules on 5,641 test frames from eight surgical videos using fixed predictions and scores. At the 10% development calibration target, DELTA retains correct partial reports for 80.0% of candidates rejected by a matched complete-or-reject baseline (5,329/6,662). Adding IV increases source-normalized weighted correct-report yield by 19.13 to 23.13 per 100 frames across all three rules relative to separately recalibrated I/IVT controls. Weighted reporting risk changes by -0.77 to +1.01 percentage points, with complete correct-triplet counts unchanged or reduced. Graded reporting makes partial surgical predictions available as localized descriptions, with an explicit trade-off between report utility and completeness.

Closest papers

Nearest neighbours by meaning across the whole index.

A Hierarchy-Aware Video-Language Model Evaluation and Hyperbolic Baseline for Surgery

cs.CV cs.AI ·2026-09-22 ·Ana Manzano Rodríguez, Pascal Mettes, Marlies P. Schijven, Cees G. M. Snoek
similarity 0.65
Abstract

Surgical procedures follow a phase-to-step hierarchy, yet the video-language models used to recognize them are evaluated with flat per-level metrics that ignore cross-level coherence and error structure. In this paper we make two contributions to address this problem, (i) we introduce SurgHiBench, the first hierarchy-aware evaluation suite for surgical video understanding, with three tasks measuring recognition, consistency, and severity across granularity levels. We evaluate a general-purpose CLIP model, a Euclidean surgical model, and, as second contribution: (ii) HyperSurg, a new hyperbolic model that enforces phase-step containment via entailment cones, across four (existing) datasets spanning three procedure types. The suite reveals that two models with the same accuracy can produce predictions of very different error severity, ranging from sibling confusions within the correct phase to unrelated cross-phase predictions. Hyperbolic geometry shifts predictions toward the correct procedural neighborhood, and these gains scale with the tree-likeness of each dataset's annotation hierarchy, providing a principled indicator when hierarchy-aware geometry helps.

0.25

Selective Surgical Action-Center Localization:Retaining Localization Gains through Supervised Scoring

cs.CV ·2026-11-08 ·Yanchong Xue, Wanting Zhang, Lingtao Yu, Guochao Zhang
hot direction similarity 0.64
Abstract

Surgical action-center localization requires both an accurate reference point and a decision about whether to report it. When frames with and without a reportable center are mixed, the benefit of improved localization depends on which correct candidates survive selection. We present a selective localization pipeline combining heatmap and coordinate supervision with a logistic-regression scorer. The scorer learns to identify predictions with a reportable center and a localization error within a specified tolerance. The internal evaluation uses 5,000 frames from 50 surgical sources, five source-grouped folds and two random seeds. Matched locator and scoring comparisons track correct candidates and their rejection. At an 80% acceptance quota per source and a tolerance of 0.10 times the image diagonal, our method retains 46.08 correct outputs per 100 inputs. The baseline locator with the same scoring approach retains 44.20. The gain is 1.88 (descriptive 95% source-paired bootstrap interval, 0.94 to 2.79). Under entropy-based selection, increased rejection of correct predictions offsets the higher supply of correct predictions on average. For the coordinate-supervised locator, supervised scoring reduces these rejections from 6.15 to 2.99 per 100 inputs. Evaluating retained correct outputs reveals whether a selection rule preserves or erodes an improvement in localization.

0.29

GRAC: Graph-Based Retrieval-Augmented Captioning for Surgical Scene Description

cs.CV cs.AI ·2026-11-06 ·Rochak Dhakal, Chen Zhao, Soup Cobb, Xin Qian, Xiaofeng Yang +1 more
benchmark similarity 0.64
Abstract · 20 Pages and 4 Figures

Purpose: Surgical operative reports support procedural documentation, surgical training, performance assessment, and workflow review. However, extracting meaningful information often requires extensive manual review and clinical expertise. Automated surgical scene descriptions could reduce this burden by leveraging recurring surgical activities and prior knowledge. Methods: We propose GRAC (Graph-Based Retrieval-Augmented Captioning), a framework that represents surgical scenes as graphs capturing relationships among instruments, actions, and anatomical structures. Given an input scene graph, GRAC learns to retrieve relevant graph-caption pairs from training data. It compares instrument-tissue interactions between retrieved examples and the input scene, using comparison features to condition retrieved-caption representations during generation. Results: On the EndoVis-18 benchmark, GRAC achieved a CIDEr-D of 5.88, compared with 4.23 for SGT and 3.36 for the strongest conventional Transformer baseline. Exact instrument-action-target triplet-set accuracy reached 52.6%, compared with 32.7% for SGT and 26.2% for the strongest conventional baseline. These results demonstrate improvements in language quality and factual agreement. On TORS, GRAC achieved a CIDEr-D of 5.06, compared with 3.80 for the strongest baseline. Conclusion: GRAC demonstrates that combining structured surgical scene representations with retrieved examples improves surgical scene captioning and produces more contextually meaningful descriptions.

0.26

Authority-Preserving Evaluation of Medical Vision-Language Assistants

cs.CV cs.AI ·2026-09-14 ·Flint Xiaofeng Fan, Cheston Tan, Yew-Soon Ong, Roger Wattenhofer
publishedbenchmark similarity 0.62
Abstract · Accepted at 2nd Emerging LLM/LMM Applications in Medical Imaging (ELAMI) 2026, held in conjunction with MICCAI 2026. To appear in the Spring…

Medical vision-language models can propose how urgently a skin lesion should be reviewed, but the local service retains authority to accept or replace that proposal under referral policy, capacity, and locally held patient context. Proposal quality and selected-action quality are therefore distinct evaluation targets, and benchmark evidence transfers between them only when local review preserves the expected action score. We introduce AuthEval, a logging and evaluation framework that records both actions, scores the selected action under declared local criteria, and, where feasible, scores the declined proposal under the same rule. It reports the resulting authority gap only when the record supports it. Because the gap is the product of the proposal-change rate and the mean score change on changed cases, that rate alone determines neither its magnitude nor its sign. On ISIC 2019, with MedGemma and simulated local review, two constraint regimes with similar change rates produced an optimistic image-equal gap under capacity ($+0.744$ simulator units) but no detectable gap under safety. The declared evaluation unit also mattered: under mixed constraints the gap reversed from $+0.374$ to $-0.206$ when weighting shifted from image to lesion-aware cluster. AuthEval thus clarifies whether a study's records support claims about the model, the workflow, or both.

0.44

RefineRank: Joint Box Refinement and Ranking for Surgical Spatio-Temporal Grounding

cs.CV cs.AI ·2026-08-25 ·Linzhe Jiang, Jiayuan Huang, Changhao Zhang, Chunyang Jiang, Zhehua Mao +1 more
theorycode/data similarity 0.62
Abstract · 17 pages, 3 figures

Surgical spatio-temporal grounding (STG) requires locating, at each video time specified by a procedural question, the object that the question asks about. Existing approaches face a trade-off: vision language models understand the question context but produce imprecise coordinates, whereas open-set detectors provide localized candidate boxes whose confidence does not reflect which box answers the question. We introduce RefineRank, which closes this gap at the candidate-box level. A compact trainable module, RefineNet, combines the language and regional features of a frozen medical vision language model with the proposals of a frozen open-set detector: it predicts a bounded coordinate correction and a quality score for every candidate box, and a fixed decoding rule returns the original or refined box with the highest score. On the MedVidBench Official Rankings (Verified), RefineRank records 0.421 STG mIoU, the highest displayed STG score, while its global multi-metric rank is 11. In a controlled evaluation on separate training and evaluation videos, coordinate correction raises the candidate oracle upper bound from 0.6772 to 0.7302, and ranking the joint pool of original and refined candidates by their RefineNet scores improves STG mIoU from 0.2719 to 0.4534, whereas separately trained selectors over the same pool reach at most 0.4186. These results show that a small box-level module can reconcile question understanding with precise localization without retraining either backbone. Code is available at [https://github.com/linzhe001/RefineRank](https://github.com/linzhe001/RefineRank).

0.32

When accuracy hides risk: risk-controlled confirmation of the critical view of safety under expert disagreement

cs.CV ·2026-11-18 ·Bowen Xiang, Xiao Yu, Mingze Xu
theory similarity 0.61
Abstract

Automated assessment of the critical view of safety (CVS) in laparoscopic cholecystectomy has advanced rapidly, but a deployed check must decide, before the first clip, whether to confirm CVS. The standard rule calibrates a threshold so that false confirmations are rare on average. We show that this rule concentrates its errors on disputed videos, which one of three expert raters judged sufficient, and that the failure is structural. When reference labels are votes of a panel sampled from a larger expert pool, the rule that maximizes sensitivity at a fixed average false-confirmation rate also maximizes false confirmation of disputed videos, and for recognizers that recover expert judgement with less noise this rate rises with accuracy. We propose certified confirmation, a risk-controlled monitoring rule whose threshold is selected by multiple testing so that, with probability at least 1 - δ, false confirmation is at most α overall, among disputed videos, within vendor and country groups, and before CVS is visible, using disputed labels only to calibrate. In a pre-specified, hash-locked evaluation on the 300 test videos (18 countries) of the SAGES CVS Challenge, standard calibration falsely confirmed 8 of 39 disputed videos (0.21; one-sided p = 0.037 against α = 0.10), whereas certified confirmation confirmed 2 (one-sided 95% upper bound 0.15). On development data, protection held for vendor and country groups left out of calibration. The cost was confirming 10% rather than 35% of sufficient videos on time. Average accuracy hides this risk; disputed cases must be protected explicitly.

0.23

Auditing Escalation Reliability in Healthcare LLM Agents: A Perturbation-Controlled Benchmark of Workflow-State Failures

cs.AI cs.CL ·2026-10-15 ·Zihao Zheng, Junyi Yao, Baichuan Li, Jiayu Long
benchmark similarity 0.61
Abstract · 8 pages, 5 figures

Trustworthy biomedical AI requires reliable action selection when evidence is missing, stale, conflicting, or late, not only accurate answers on static inputs. We instantiate an episode-level evaluation protocol as 400 synthetic healthcare workflow episodes: 40 hand-authored semantic seeds across four task classes, two surface parameterizations, and five controlled variants, scored on ANSWER/ASK CLARIFICATION/ABSTAIN/ESCALATE under a pre-declared 5:1 missed-versus-unnecessary escalation cost. Reference actions are single-author study-policy labels, not clinician-adjudicated ground truth. Across five open-weight configurations (1.7B--9.7B), clean decision accuracy reaches 0.97, yet uncertainty-prompted models miss 32--97% of required escalations. The stronger 9.7B configuration narrows but does not remove the gap, falling from 0.97 clean accuracy to 0.78 under conflict. Verbalized confidence is inconsistent (ECE 0.10--0.54; failure AUROC 0.48--0.77), and a development-tuned confidence-only policy yields no utility gain. A transparent workflow-aware controller improves 5:1 utility for every model and reduces missed escalations to 0.12--0.24, at specificity 0.83--0.93. However, a 30-concept lexical stress test reduces each rule detector's recall from 1.00 on trigger phrases to 0.00 on unseen paraphrases, limiting the controller claim to this benchmark. Seed-clustered statistics, ablations, cross-task transfer, raw outputs, and manifests are packaged for audit. The results motivate explicit workflow-state evaluation while showing that apparently effective routing controls need their own generalization tests.

0.46

Beyond Endpoint Gains: A Weight-Delta Audit of Medical Specialization

cs.AI ·2026-08-21 ·Praphul Singh, Shanu Kumar, Akshat Agarwal
top venuebenchmarkhot direction similarity 0.61
Abstract · Preprint: EMNLP 2026

Specialist language models are usually understood through endpoint gains: the generalist scores lower, the specialist scores higher, and the difference is treated as evidence of specialization. This leaves the released update itself largely unexamined. We propose a paired weight-delta path audit and apply it to two public, aligned generalist-to-medical-specialist checkpoint pairs: Gemma-3-4B-IT to MedGemma-4B-IT and Qwen2.5-7B-Instruct to HuatuoGPT-o1-7B. In both pairs, the full decoder-side update strongly reconstructs measured medical benchmark movement (0.974 and 1.183 endpoint-normalized retention), making each decoder delta an appropriate substrate for the audit. Yet the movement is not cleanly localized. MLP is the strongest broad component family in both pairs, but mixed off-domain movements, 10-seed matched controls, and endpoint-anchored rollbacks prevent a unique coarse-family explanation. The audit therefore separates update-level reconstruction from component-level explanation. Its claims concern text-only multiple-choice benchmark movement, not clinical validation, repair, or circuit-level mechanism.

0.50