CS Paper Radar

CEM-TUDASR: Computationally efficient multi-modality transformer based unsupervised domain adaptive super-resolution approach

Anjali Sarvaiya, Jay Kadel, Kishor Upla, Kiran Raja

cs.CV ·submitted 2026-09-10 08:10 UTC · v1
published

Abstract

Wireless Capsule Endoscopy (WCE) enables non-invasive visualization of the gastrointestinal tract, but its miniaturized optics, sensor limitations, and wireless transmission constraints result in low-resolution images with reduced visibility of diagnostically important structures. This paper proposes CEM-TUDASR, a computationally efficient unsupervised Transformer-based super-resolution framework for WCE image enhancement without paired low-resolution (LR) and high-resolution (HR) training data. A domain-adaptive degradation network synthesizes realistic WCE-like LR images from HR conventional endoscopy images, reducing the domain gap and enabling effective unpaired learning. The SR generator integrates Deep Attention Blocks (DABs) and a Fusion Attention Block (FAB) to capture long-range contextual dependencies and fine local structures while preserving perceptual and structural fidelity. The model is trained on a curated dataset derived from Kvasir Capsule and evaluated on KID and GIANA for cross-dataset generalization. No-reference quality metrics, including BRISQUE, PIQE, NIQE, and the domain-specific EndoQM, show that CEM-TUDASR consistently outperforms existing unsupervised SR methods. Qualitative results further demonstrate improved restoration of mucosal textures, vascular patterns, and clinically relevant anatomical details. Cross-domain experiments on retinal images additionally demonstrate the adaptability of the framework. With only 2.67 million parameters and 169.94 GFLOPs, CEM-TUDASR achieves high-quality reconstruction while maintaining computational efficiency, making it suitable for resource-constrained clinical and embedded endoscopic applications.

Comment
Published in Biomedical Signal Processing and Control, Volume 129, 2027, Article 111315
Journal ref
Biomedical Signal Processing and Control, Volume 129, 2027, Article 111315
DOI
10.1016/j.bspc.2026.111315

Closest papers

Nearest neighbours by meaning across the whole index.

UnCapsTSR: An Unsupervised Transformer-based Image Super-Resolution Approach for Capsule Endoscopy Images

cs.CV ·2026-09-02 ·Anjali Sarvaiya, Shubh Kawa, Lalit Agrawal, Jagrit Joshi, Kishor Upla +1 more
publishedhighly cited similarity 0.79
Abstract · Accepted manuscript of the article published in Neurocomputing, Volume 665, Article 132161, 2026

Wireless Capsule Endoscopy (WCE) captures and streams video while passing through a patient's Gastrointestinal (GI) tract and is used to examine its irregularities. Although advantageous over conventional endoscopy, WCE suffers from limitations related to capsule size and wireless transmission, resulting in images with coarser resolution. This work presents UnCapsTSR, an unsupervised transformer-based Generative Adversarial Network (GAN) framework for improving the spatial resolution of Low-Resolution (LR) WCE images. The proposed method accomplishes SR without explicit degradation estimation of real-world LR data and eliminates the need for true LR-HR pairs. UnCapsTSR employs a Bilateral Total Variation (BTV) loss to ensure spatial continuity in SR images. A newly curated dataset from the Kvasir Capsule dataset is also presented for training WCE SR models. Generalizability is validated on KID and GIANA datasets that are not used during training. A new non-reference metric, Endoscopy Quality Metric (EndoQM), is introduced for quantitative evaluation of domain-specific WCE data. Experiments demonstrate consistent improvement over state-of-the-art unsupervised SR approaches using NIQE, BRISQUE, PIQE, and EndoQM. Statistical evaluation shows 40 to 80 percent improvement in EndoQM from LR to SR across the evaluated datasets.

0.66

Rethinking Evaluation for Video Capsule Endoscopy: Validation Protocol, Source-Video Diversity, and Cross-Dataset Shift

cs.CV ·2026-11-09 ·Smriti Regmi, Debesh Jha
benchmark similarity 0.70
Abstract · 4 figures and 9 tables

Automated video capsule endoscopy (VCE) classification remains challenging due to severe class imbalance, limited source-video diversity, and variations in image appearance across datasets. Although deep learning models have demonstrated promising performance on established benchmarks, their reliability under different validation protocols and external dataset shift remains insufficiently understood. In this work, we introduce DMSNet, a deformable adaptive multi-scale edge-guided attention network integrating adaptive multi-scale feature fusion, deformable lesion refinement, edge-guided attention, and sparse feature selection. We benchmark DMSNet against 18 representative convolutional, transformer-based, and pretrained vision architectures on Kvasir-Capsule under random frame-level and source-video-disjoint validation protocols. We further identify source-video overlap between the original training and test splits and evaluate model performance on both the complete test set and a source-video-disjoint subset. Under random frame-level validation, DMSNet achieves an MCC of 0.442 $\pm$ 0.020 on complete test set and 0.456 $\pm$ 0.037 on the source-video-disjoint subset. On the external CV2024 dataset, ConvNeXt-Tiny achieves the highest MCC of 0.207, while performance degrades substantially across architectures relative to in-domain evaluation. We further investigate source-video diversity using a per-class cap of 200 training frames. With the total training-set size held fixed, increasing the maximum source-video coverage from four to eight improves MCC for 17 of 19 evaluated architectures. Overall, these findings demonstrate that model rankings and classification performance depend substantially on evaluation protocol, training-source diversity, and cross-dataset shift, emphasizing the need for source-aware validation and rigorous external evaluation of VCE classification systems.

0.22

EndoFSA: Endoscopic Few-Shot Image Generation via Rank-Constrained Parameter Adaptation

cs.CV ·2026-09-24 ·Panagiota Gatoula, Grigoris Karypidis, Dimitris K. Iakovidis
publishedbenchmark similarity 0.70
Abstract · Presented at the 39th IEEE International Symposium on Computer-Based Medical Systems (CBMS 2026), June 2026

WCE produces large-scale gastrointestinal image data yet pathological findings remain significantly underrepresented limiting the generalization performance of deep-learning based abnormality detection systems. SDG methods offer a practical solution to mitigate this imbalance. However their training directly on scarce abnormal samples often results in instability overfitting and structural distortions. Addressing these challenges requires controlled adaptation mechanisms that preserve anatomical priors while enabling realistic pathological variation. This paper presents EndoFSA a GAN-based model for Endoscopic Few-Shot image generation by Adaptation in WCE imaging. EndoFSA leverages a generator pretrained on abundant normal data and adapts it to abnormal domains using limited number of training samples through a rank-constrained parameter adaptation where only a small number of modulation parameters is updated while the pretrained weights remain frozen. By restricting parameter updates to a low dimensional subspace and incorporating perceptual boundary regularization and cluster-wise diversity control EndoFSA enables efficient model adaptation under limited data conditions and mitigates mode collapse while preserving the anatomical priors learned from normal data. Importantly EndoFSA operates without requiring pixel-level annotations, masks or bounding box supervision. Evaluation on publicly available WCE benchmark datasets spanning various abnormal categories demonstrates that EndoFSA generates abnormal images reproducing real lesions morphology. Moreover in a downstream classification task training an image classifier solely on synthetic abnormal images generated by EndoFSA yields performance comparable to that obtained with real images.

0.21

MultiAttenGastro: Multi-Dimensional Attention Augmentation for Gastrointestinal Endoscopy Classification

cs.CV ·2026-09-04 ·Sadhana Devarajan, Praveen Kumar Chandaliya, Dhruvin Jashvant Kumar Shah, Kishor Upla, Kiran Raja
benchmark similarity 0.69
Abstract · 17 Pages, 2 Figures, CVIP2026

Automated gastrointestinal (GI) endoscopy classification requires models that generalize across diverse modalities and class distributions, often far from natural-image pretraining. We propose MultiAttenGastro, a plug-and-play attention framework with parallel 1-D channel, 2-D spatial, and 3-D contextual heads, and present the first systematic cross-dataset evaluation across eight CNN and transformer backbones on five public GI datasets (80 backbone--dataset runs). We find that attention effectiveness is not universal but tracks the representational gap between ImageNet features and the target distribution: MultiAttenGastro improves 6 of 8 backbones on Kvasir-Capsule (14-class WCE, large gap; best macro F1 98.33\%), is uniformly negative on the small-gap Kvasir-v2 benchmark (0/8), and shows mixed outcomes on datasets with intermediate gap. Five-seed ablation on the strongest case (Kvasir-Capsule, ConvNeXt-Tiny) shows this improvement is directionally consistent, but not statistically decisive (paired $t$: $p=0.47$; Wilcoxon: $p=0.63$), and that individual attention heads are not uniformly beneficial in isolation only their combination yields a positive mean effect. Centered Kernel Alignment (CKA) analysis links this pattern to representational redundancy: low inter-head CKA under large domain gaps coincides with the framework's only consistent gains, while high redundancy under small gaps coincides with its losses. We report these results, including the non-significant margins, as evidence for when and why multi-dimensional attention helps GI endoscopy classification, rather than as a claim that MultiAttenGastro is a strictly superior architectural choice.

0.32

FreeTransformSR: Efficient Lightweight Image Super-Resolution via Free Low-Rank Learnable Transform

cs.CV ·2026-09-05 ·Hongji Li, Yunhui Li, Jianing Wang, Xiaodong Wan
benchmark similarity 0.65
Abstract · 16 pages, 2 figures

Single image super-resolution aims to reconstruct high-resolution images from low-resolution inputs. This paper proposes FreeTransformSR, a novel lightweight super-resolution network based on a channel-wise free low-rank learnable transform. The transform learns task-adaptive basis functions in a data-driven manner, enabling adaptive feature modulation with minimal parameter overhead. To further enhance high-frequency detail recovery, we in-troduce a local feature modulation branch that complements transform-domain processing with depthwise convolu-tion. In addition, a soft complexity adaptive module dynamically fuses the outputs of local convolution and window self-attention branches through a lightweight gating network, adaptively adjusting the fusion ratio based on regional texture characteristics. An adaptive intensity modulation strategy is also incorporated to adjust transform-domain response strength at the sample level, enabling the network to dynamically adjust processing intensity according to input features. Extensive experiments on five benchmark datasets demonstrate that FreeTransformSR achieves competitive PSNR/SSIM performance with significantly fewer parameters and FLOPs. Specifically, FreeTransformSR achieves 32.41 dB on BSD100 (X2, 634K parameters) and 27.00 dB on Urban100 (X4, 653K parameters), while delivering faster inference speed than competing methods, making it well-suited for deployment in resource-constrained scenarios.

0.25

Attention Residual U-Net for Enhanced Cross-Dataset Polyp Localization in Endoscopy

cs.CV ·2026-11-02 ·Romil Imtiaz, Dimitris K. Iakovidis
similarity 0.64
Abstract

Precise localization of pathologies in endoscopy is a critical task in medical imaging, with deep learning (DL) frameworks now serving as the primary methodology for automated lesion detection. Today, generic object detection methods, such as YOLO, provide state-of-the-art performance in this context. However, their robustness often decreases under real-world clinical conditions due to variations across endoscopic devices, intestinal content, dataset bias, and the presence of small or low-contrast lesions. Motivated by these challenges, we propose a novel polyp detection framework based on a Attention Residual U-Net offering enhanced polyp localization. Instead of directly regressing bounding boxes from global feature representations, the proposed method learns lesion-focused spatial representations and converts them into bounding-box predictions during inference. Unlike conventional encoder-decoder localization frameworks that directly propagate all encoder activations into the decoding pathway, the proposed method selectively suppresses irrelevant background responses before feature reconstruction. This encourages the network to focus on lesion-relevant spatial evidence under heterogeneous imaging conditions. To achieve this, the framework integrates residual feature propagation, attention-guided encoder filtering, and multi-scale contextual learning. The proposed framework was evaluated on multiple public endoscopic datasets acquired using different devices. The comparative experimental results validate its effectiveness over the relevant state-of-the-art methods, with an improvement up to 22.0% in terms of the area under the receiving operating characteristic (AUC).

0.18

MOSAIC: Multi-Scale Object-Centric Slot Adaptation with Image-Guided Prototype Conditioning for Remote Sensing Image Super-Resolution

cs.CV ·2026-10-09 ·Liuqian Wang, Hanyu Xie, Rongzhen Zhao, Qi Ming, Chunjie Ma +4 more
similarity 0.63
Abstract

Pretrained diffusion Transformers provide powerful generative priors for image restoration, yet adapting them to structure-sensitive tasks remains challenging. Existing methods mainly rely on dense low-resolution features as conditions, which often entangle salient objects with dominant backgrounds and provide limited abstraction for recurring structures across scales and samples. We propose MOSAIC, an object-centric representation learning framework that learns compact structural conditions for generative model adaptation. MOSAIC organizes degraded observations into structural slots that capture complementary object-level and layout-level evidence, integrates them across resolutions, and retrieves recurring priors from a shared prototype memory to complement incomplete image-specific information. The resulting representations are injected into selected layers of a frozen diffusion Transformer through lightweight residual modulation, preserving its pretrained generative prior. We instantiate MOSAIC for remote sensing image super-resolution, where small objects, repetitive structures, and large-scale variation provide a challenging testbed. Experiments on UCMerced, AID, and RSSCN7 show consistent PSNR improvements across $\times 2$ and $\times 4$ settings and favorable LPIPS results at $\times 4$. Qualitative results further demonstrate improved preservation of small objects, structural boundaries, and repetitive spatial patterns.

0.19

Unsupervised Multi-modal Medical Image Registration via Invertible Translation

cs.CV ·2026-11-17 ·Mengjie Guo, Jinming Duan
code/data similarity 0.63
Abstract

In medical imaging, the alignment of multi-modal images plays a critical role in providing comprehensive information for image-guided therapies. Despite its importance, multi-modal image registration poses significant challenges due to the complex and often unknown spatial relationships between different image modalities. To address this, we introduce a novel unsupervised translation-based multi-modal registration method, termed Invertible Neural Network-based Registration (INNReg). INNReg consists of an image-to-image translation network that converts multi-modal images into mono-modal counterparts and a registration network that uses the translated mono-modal images to align the multi-modal images. Specifically, to ensure the preservation of geometric consistency after image translation, we introduce an Invertible Neural Network (INN) that leverages a dynamic depthwise convolution-based local attention mechanism. Additionally, we design a novel barrier loss function based on Normalized Mutual Information to impose constraints on the registration network, which enhances the registration accuracy. The superior performance of INNReg is demonstrated through experiments on two public multi-modal medical image datasets, including MRI T1/T2 and MRI/CT pairs. The code is available at https://github.com/MeggieGuo/INNReg

0.42