CS Paper Radar

Pixel-KWS: Rendered-Glyph Representations for Orthographically Robust Open-Vocabulary Keyword Spotting

Adarsh Arigala, Ramesh Gundluru, Srinivasan Umesh, Ludovick Lepauloux, Hakim Hacid

eess.AS cs.AIcs.CVcs.SD ·submitted 2026-11-14 12:53 UTC · v1

Abstract

Open-vocabulary keyword spotting (KWS) allows users to enroll arbitrary keywords at inference time. However, character and phoneme-based text encoders rely on fixed symbol vocabularies learned during training, limiting their ability to handle unseen or visually similar symbols such as homoglyphs, l33tspeak substitutions, and accented characters. We propose PIXEL-KWS, an open-vocabulary KWS framework that represents characters through rendered glyphs rather than learned embedding lookups. We render each character as a grayscale image and encode it with a lightweight CNN, enabling the text encoder to generalize from visual form rather than a fixed vocabulary. On LibriPhrase, PIXEL-KWS and its matched character-based counterpart achieve 9.57% and 9.55% EER on LibriPhrase-hard, 0.43% and 0.41% EER on LibriPhrase-easy, outperforming all published baselines without pre-training or synthetic data. PIXEL-KWS provides substantially greater robustness to both homoglyph and l33tspeak symbol-level perturbations. These results highlight glyph-based representations as a simple and effective approach for robust open-vocabulary KWS in practical settings.

Comment
5 pages, 4 figures, 3 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Unsupervised Domain Adaptation for Symbol Spotting in Historical Encrypted Manuscripts

cs.CV ·2026-09-07 ·Giuseppe De Gregorio, Alicia Fornés, Lei Kang, Beáta Megyesi
similarity 0.68
Abstract

The decipherment of historical encrypted manuscripts poses a fundamental challenge in Digital Humanities: before any transcription can begin, the symbol inventory of the underlying cipher alphabet must first be identified and characterized. We address this challenge through symbol spotting: given a candidate alphabet specified as a set of rendered font glyphs, the task is to determine whether and where its characters appear in an unseen handwritten document, without any labeled examples from the target script. The main difficulty lies in the domain gap between clean, digitally rendered font queries and degraded handwritten manuscript symbols. We propose a three-stage pipeline that bridges this gap without manual annotation, combining a joint SimCLR+DANN encoder for domain-invariant glyph representations with an embedding-space style-adaptation mechanism applied at retrieval time, requiring no re-training. Experiments on fourteen pages from seven encrypted manuscript collections show that our method outperforms zero-shot foundation models, including CLIP and DINOv2, by a large margin ($+0.194$ P@1 over CLIP ViT-L/14), and surpasses task-specific trained baselines by $+0.138$ P@1. We further demonstrate that the Raw-Cover metric, computed in a fully unsupervised setting, provides a meaningful script-family fingerprint that identifies the underlying alphabet of an unknown document. This capability is of direct practical relevance to palaeographers, historians, and other researchers working with undeciphered manuscripts.

0.19

GlyphAnchor: Enhancing Visual Text Rendering via Position-Anchored Glyph Priors

cs.CV ·2026-09-02 ·Qiang Xiang, Shuang Sun, Binglei Li, Yibo Chen, Xu Tang +2 more
benchmark similarity 0.65
Abstract

Rendering accurate text remains difficult for image generation and editing models, especially when the target contains long, complex, and densely arranged text or rare characters. Existing approaches either improve native text rendering through stronger backbones and data-centric training without explicit glyph priors, or incorporate glyph priors through specialized designs that remain insufficiently accurate and robust under challenging scenarios. We introduce GlyphAnchor, a novel text-rendering enhancement method for both text-to-image and image-editing diffusion transformer models. GlyphAnchor enhances the backbone with lightweight glyph patch conditions whose positions are anchored to the target image through the model's native positional encoding. We train this capability with staged supervised finetuning and further refine it with text-aware post-training to improve robustness. We also introduce InfoTextBench, a benchmark for evaluating text-rich visual text rendering in both generation and editing settings. Experiments across multiple backbones and benchmarks, including long, complex, and densely arranged text and rare character scenarios, show that GlyphAnchor consistently improves text fidelity while preserving overall image quality.

0.33

CORD-KWS: Calibrated, Order-Aware Detection for Open-Vocabulary Keyword Spotting

cs.SD eess.AS ·2026-09-25 ·Ramesh Gundluru, Adarsh Arigala, Sri Rama Murty Kodukula
similarity 0.64
Abstract · 5 pages, 2 figures

Open-vocabulary keyword spotting (KWS) must detect arbitrary keywords without retraining. Cross-attention-based models achieve state-of-the-art performance but require pairwise interaction between the audio and each keyword, recomputing the fused representation for every audio--keyword pair. Embedding-based models avoid this computational cost through independent encoding and similarity scoring, yet remain inferior on challenging benchmarks such as LibriPhrase-hard. We show that this gap is primarily a training issue rather than a limitation of model capacity. While contrastive learning encourages correct keywords to rank above negatives, it does not explicitly calibrate absolute similarity scores, which are critical for applying a fixed threshold across keyword detection. We propose Calibrated, Order-aware Detection KWS (CORD-KWS), a training framework that introduces two complementary objectives while retaining the same encoders, cosine scoring, and O(1) keyword enrollment: a calibrated detection head that supervises absolute scores, and a frame-level CTC objective that preserves phonetic order information. CORD-KWS achieves EERs of 0.43% and 9.64% on LibriPhrase-easy and LibriPhrase-hard, respectively, outperforming existing embedding- and cross-attention-based models.

0.26

Enhanced Video Text Editing with Trajectory-Aligned Glyph Rendering

cs.CV ·2026-09-28 ·Shulian Zhang, Xiangyu Shu, Wenbo Li, Jian Chen, Yong Guo
benchmark similarity 0.63
Abstract

Video text editing aims to replace or add text in a video while keeping the rest of the video unchanged, which requires the edited text to be correct in every frame and to move coherently with the scene. Despite the remarkable progress of video diffusion models, they struggle to reproduce exact stroke structures and often produce garbled or wrong characters, especially for characters with complex strokes. To address this, we propose a trajectory-aligned glyph rendering reference that provides explicit per-frame glyph guidance following the position and perspective of the text, and a depth-normalized recognizer feature supervision that supervises the generated text on multi-depth features of a frozen text recognizer with per-depth normalized errors, targeting stroke errors overlooked by the diffusion loss. We further build VTEdit, a benchmark of 288 real-scene clips with 440 annotated text trajectories covering text replacement and text addition, which will be publicly released to facilitate future research. Experiments on VTEdit show that our method outperforms image text editing methods, video editing methods, and commercial models in text accuracy and background preservation, achieving a sentence accuracy of 0.9408, and receives the highest preference in a user study.

0.26

DDVR-Net: Degradation-Aware Decoupling and Visibility-Guided Restoration for Historical Document Recognition

cs.CV ·2026-10-12 ·Jiu Sha, Yu Weng, Mengxiao Zhu
top venuebenchmarkcode/data similarity 0.63
Abstract · 18 pages. Accepted to the 34th ACM International Conference on Multimedia (ACM MM 2026)

Historical document analysis is crucial for cultural heritage preservation and low-resource language digitization. However, real historical documents often suffer from severe degradation, such as ink erosion and structural damage, causing mixed-visibility character sequences. Conventional OCR systems struggle in such evidence-limited scenarios, while contextual language models may over-correct visually clear characters. To address these challenges, we propose DDVR-Net, a degradation-aware decoupling and visibility-guided restoration framework for historical document recognition. DDVR-Net decouples intrinsic glyph structures from physical corruption at the feature level and predicts local visibility to estimate character reliability. A contextual decoder is selectively activated through a visibility-guided gate, restoring degraded characters while preserving clear visual recognition. Furthermore, we construct a low-resource multilingual benchmark covering Tibetan, Mongolian, and Uyghur scripts with 187000 document pairs, combining large-scale synthetic data and progressive training with real-world annotations to reduce the synthetic-to-real gap. Extensive experiments demonstrate that DDVR-Net consistently outperforms existing methods across three languages and diverse reading orders, providing an effective framework and benchmark for low-resource historical document recognition. Code and Benchmark: https://github.com/Shajiu/DDVR-Net

0.47

The Pen is Mightier than the Pixel: Natural Textual Unlearnable Example for Large Vision Language Models

cs.AI ·2026-11-03 ·Chenyu Yi, Jiawen Zong, Sami Hormi, Bingquan Shen, Xudong Jiang +1 more
similarity 0.62
Abstract

The proliferation of Large Vision Language Models (LVLMs), trained on vast and often sensitive datasets, raised concerns regarding data privacy and unauthorized usage. Unlearnable Examples (UEs), which subtly perturb training data to mislead the models, have emerged as a promising defense method. This paper presents the first exploration of UEs specifically on LVLMs, which poses unique challenges due to their inherent multimodal nature. We propose a novel text-based perturbation method, Natural Textual Unlearnable Examples (NTUE), which can effectively mislead LVLMs by subtly manipulating the image label, establishing a shortcut between an identity of interest and a separate target, without introducing factual misinformation or affecting readability. Comprehensive experiments compared with traditional UE methods demonstrate the superior performance of NTUE. Our findings show that NTUE achieves a high success rate with a poison rate as low as 5%, underscoring its potential as a practical and robust solution for protecting multimodal data in the era of LVLMs.

0.08

Beyond Clean Accuracy: The Practical Challenges and Benefits of Pixel-Based Language Models

cs.CL ·2026-10-16 ·Haowen Tang, Wei Xia, Raoyuan Zhao, Hinrich Schütze, Yihong Liu
similarity 0.61
Abstract · preprint

Pixel-based language models process rendered text without relying on a fixed subword vocabulary, offering a promising approach to noisy and multilingual inputs. However, their practical value depends on whether they can be reliably adapted to downstream tasks. To this end, we systematically examine this question through fine-tuning behavior, typographical robustness, and zero-shot crosslingual transfer. Under our tested settings, pixel-based models require longer training and broader labelled-data coverage than text-based models, while delayed learning and learning-rate-dependent failures make adaptation less reliable. Our analyses further suggest that task-relevant information is less readily accessible in pretrained pixel representations, placing greater demands on fine-tuning. Once successfully adapted, however, pixel-based models lose substantially less accuracy under typographical noise and achieve higher target-language accuracy than English-pretrained text-based counterparts on two crosslingual benchmarks, despite weaker clean English performance. Together, these findings reveal a central trade-off: replacing tokenization with rendering reduces sensitivity to corrupted and unfamiliar inputs, but shifts a central bottleneck to downstream adaptation.

0.15

Query-adaptive Recurrent Tokenization

cs.CV ·2026-10-16 ·Bavesh Balaji, Pranav Gopinath, Pouya Sadeghi, Sirisha Rambhatla
similarity 0.61
Abstract

Visual information worth extracting from an image and the token budget it warrants depend on the text query. Yet modern vision and vision-language models largely encode images independently of the query and allocate a fixed visual token budget to every input. We introduce the Query-Adaptive Recurrent Tokenizer (QART), a backbone-agnostic module that makes visual representations both query-steerable and budget-adaptive. QART steers the visual representation toward query-relevant evidence while adapting its token budget to the amount of information required. Its key principle is simple. Additional tokens are allocated until the visual features relevant to the query are represented with sufficient fidelity. Starting from frozen encoder features, QART recurrently constructs a compact representation and uses reconstruction quality over the query-relevant visual support to control token allocation. On Visual7W and VQAv2, QART achieves 81.96% and 76.37% accuracy using only about 29 visual tokens, reducing token counts by 20-25x relative to dense VLM inputs. It further reaches 92.0% P@1 on CORE and 36.3% PR-AUC on PODS, outperforming SteerViT on PODS by 8.4%. Together, these results show that visual representations need not be fixed in either content or capacity and can instead adapt to what each query requires.

0.18