CS Paper Radar

STEPS: Scene Text Editing with Preserved Style Using Diffusion and Contrastive Style Encoding

Nicolas Thiebaut, Nameer Hirschkind, Xiao Yu, Kyle Spence

cs.CV cs.LG ·submitted 2026-09-29 22:52 UTC · v1
published

Abstract

We introduce Scene Text Editing with Preserved Style (STEPS), a novel diffusion model architecture for quality text replacement in images. Scene Text Editing (STE), also known as Visual Text Editing, consists of changing the textual content in an image while conserving the original style, e.g. font, colors, orientation, background, etc. STEPS advances the state of the art in STE through directed focus on improved style preservation. We introduce a style encoder for visual text that captures style independently of textual content, and a model architecture that combines the style encoder with multiple semantic conditions (target text characters encoding and rendered glyphs). STEPS achieves superior results to previous STE methods in style preservation, output readability, and subjective quality.

Comment
9 pages, 5 figures, 4 tables. A version of this paper appeared in PAKDD 2026 (LNCS vol. 16618)
Journal ref
Data Science: Foundations and Applications (PAKDD 2026), Lecture Notes in Computer Science, vol. 16618, pp. 473-484, Springer (2027)
DOI
10.1007/978-981-92-1947-6_38

Closest papers

Nearest neighbours by meaning across the whole index.

GlyphAnchor: Enhancing Visual Text Rendering via Position-Anchored Glyph Priors

cs.CV ·2026-09-02 ·Qiang Xiang, Shuang Sun, Binglei Li, Yibo Chen, Xu Tang +2 more
benchmarkhighly cited similarity 0.70
Abstract

Rendering accurate text remains difficult for image generation and editing models, especially when the target contains long, complex, and densely arranged text or rare characters. Existing approaches either improve native text rendering through stronger backbones and data-centric training without explicit glyph priors, or incorporate glyph priors through specialized designs that remain insufficiently accurate and robust under challenging scenarios. We introduce GlyphAnchor, a novel text-rendering enhancement method for both text-to-image and image-editing diffusion transformer models. GlyphAnchor enhances the backbone with lightweight glyph patch conditions whose positions are anchored to the target image through the model's native positional encoding. We train this capability with staged supervised finetuning and further refine it with text-aware post-training to improve robustness. We also introduce InfoTextBench, a benchmark for evaluating text-rich visual text rendering in both generation and editing settings. Experiments across multiple backbones and benchmarks, including long, complex, and densely arranged text and rare character scenarios, show that GlyphAnchor consistently improves text fidelity while preserving overall image quality.

0.35

Efficient Text-to-Image Generation: An Adaptive Step Schedule Controller for Diffusion Models

cs.CV cs.AI ·2026-09-15 ·Kuluhan Binici, Cihan Acar, Shivam Aggarwal, Siying Liu, Tulika Mitra
published similarity 0.70
Abstract

Text-to-image diffusion models often use a fixed number of denoising steps, balancing time costs and image quality. However, the optimal number of steps depends on the complexity of the input text prompt. We propose an adaptive diffusion controller that dynamically adjusts the number of steps to generate high-quality images efficiently, without additional model training. By leveraging a mixture of step schedules with varying step sizes and evaluating the error term discrepancy at each timestep, our method transitions between schedules to optimize performance. Experiments on COCO and DiffusionDB show that our approach reduces inference time while maintaining visual fidelity, offering a more efficient alternative for text-to-image diffusion models.

0.17

Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?

cs.CV ·2026-10-20 ·Xingsong Ye, Yongkun Du, Jiaxin Zhang, Chong Sun, Zhineng Chen
top venuecode/data similarity 0.68
Abstract · Accepted as a conference paper at NeurIPS 2026

Aiming to modify textual content while preserving the original style, Scene Text Editing (STE) has significant practical value in many applications. STE typically requires paired images before and after editing for training. However, recent approaches rely heavily on synthetic paired data and complex disentanglement strategies. They struggle to faithfully capture the diversity and complexity of natural scenes, often producing artifacts or hallucinations. In this work, we challenge the common practice and investigate whether STE can be driven by turning each real sample into its own paired supervision. Concretely, we explore multiple ways to disrupt the original image (e.g., Crop, Shuffle, Mask) to construct self-paired training data, and build a strong baseline (TextRIRO}) upon a conditional diffusion architecture. Our experiments show that combining character-column shuffling with image cropping (Shuffle&Crop) effectively destroys text semantics while preserving key style cues, enabling the model to learn editing behaviors directly from the real text image. With this concise yet real-oriented design, TextRIRO achieves promising performance in both accuracy and visual realism across real-world benchmarks, substantially mitigating the synthetic-to-real domain gap. Furthermore, we leverage TextRIRO to build TextRIRO-3M, a large-scale realistic dataset for long-text recognition by concatenating edited and original images. Experimental results show that training recognition models with TextRIRO-3M significantly improves recognition accuracy, demonstrating that TextRIRO provides highly useful training resources for downstream tasks. Overall, TextRIRO lifts STE to realistic, reliable, and widely applicable new levels. Code is available at https://github.com/YesianRohn/TextRIRO.

0.75

Enhanced Video Text Editing with Trajectory-Aligned Glyph Rendering

cs.CV ·2026-09-28 ·Shulian Zhang, Xiangyu Shu, Wenbo Li, Jian Chen, Yong Guo
benchmark similarity 0.68
Abstract

Video text editing aims to replace or add text in a video while keeping the rest of the video unchanged, which requires the edited text to be correct in every frame and to move coherently with the scene. Despite the remarkable progress of video diffusion models, they struggle to reproduce exact stroke structures and often produce garbled or wrong characters, especially for characters with complex strokes. To address this, we propose a trajectory-aligned glyph rendering reference that provides explicit per-frame glyph guidance following the position and perspective of the text, and a depth-normalized recognizer feature supervision that supervises the generated text on multi-depth features of a frozen text recognizer with per-depth normalized errors, targeting stroke errors overlooked by the diffusion loss. We further build VTEdit, a benchmark of 288 real-scene clips with 440 annotated text trajectories covering text replacement and text addition, which will be publicly released to facilitate future research. Experiments on VTEdit show that our method outperforms image text editing methods, video editing methods, and commercial models in text accuracy and background preservation, achieving a sentence accuracy of 0.9408, and receives the highest preference in a user study.

0.26

Efficient Scene-Text Refinement via Pixel-Space Grouped Image Generation

cs.CV ·2026-10-13 ·Yaoli Liu, Guo-Hua Wang, Yiyang Chen, Qing-Guo Chen, Zhao Xu +2 more
benchmark similarity 0.68
Abstract · 29 pages, including appendices

Generated images often have coherent layouts and visual content, yet local errors in small text and complex characters still limit their readability and practical use. Scene-text refinement requires accuracy, preservation, and efficiency. Existing methods based on variational autoencoders (VAEs) can lose fine strokes during reconstruction, while full-image editing can shift background colors. In this work, we introduce PixText, a model that formulates local text refinement as grouped image generation in pixel space. Conditioned on the original image, PixText generates only the target regions, preserves unedited content, and uses MeanFlow for four-step generation. We further introduce Text Refinement Bench to evaluate text correction and image preservation in generated images. On this benchmark, PixText achieves 83.06% glyph pass and 94.66% exact target-character OCR accuracy, versus 36.77% and 52.31% for the strongest baseline. Its four-step MeanFlow variant retains an 81.84% pass rate with less than one GPU-second of amortized generation cost per image.

0.37

MOSS-Image Technical Report

cs.CV ·2026-10-16 ·MOSI. AI
code/datahot directioncommunity pick similarity 0.67
Abstract · 35 pages. GitHub: https://github.com/OpenMOSS/MOSS-Image

Recent closed-source image generation systems can produce diverse visual content from simple compositions to information-dense graphics, yet open-weight text-to-image models still struggle with dense text and intricate layouts. To narrow this gap, we introduce MOSS-Image, an open generation system that combines high-fidelity visual synthesis with intent-aware composition. First, we pretrain a 5B single-stream diffusion transformer (DiT) from scratch with a carefully designed recipe, enabling bilingual long-form text rendering and precise layout control. Second, we develop an intention-aware agentic prompt enhancement (PE) module that translates diverse user requests into detailed content and layout specifications. Together, they enable MOSS-Image to generate text-rich and layout-coherent images. We further introduce LongLongTextBench (LLTB) for evaluating long-form English and Chinese text rendering, and RealLayoutArena (RLA) for end-to-end layout-rich generation. MOSS-Image achieves state-of-the-art performance among open-weight models on LLTB, with an average text coverage of 96.3%. On RLA, MOSS-Image ranks second in the end-to-end system comparison, while its backbone ranks first among generators evaluated with shared enhanced prompts. We will release the model, training recipe, and evaluation framework to support further research and promote the text-to-image community.

0.53

Disentangled Stylized Image Generation via IP-Cross-Attention Rearrangement

cs.CV ·2026-11-07 ·Beibei Dong, Yueming Lyu, Yue Jiang, Jing Dong
similarity 0.66
Abstract · 12 pages, 11 figures

With a solid theoretical foundation and pretraining on large-scale datasets, diffusion-based methods have achieved controllable generation leveraging text and image conditions, and generating excellent stylized images is possible using diffusion models. However, previous methods face significant challenges. First, because diffusion generation is iterative, the common approach of learning implicit style representations via cycle-consistency strategies is too costly and not well-suited to diffusion, even for LoRA-based methods. Second, decoupling style is difficult in the latent space of diffusion models. Representations of attributes are coupled in the latent space of diffusion models; thus, extracting style attributes with diffusion models usually suffers from the issue of content leakage. Furthermore, because style is hard to represent in natural language, capturing style using text prompts or textual inversion is deficient. In this paper, we propose a method for achieving disentangled stylized image generation through IP-cross-attention rearrangement, based on our experimental observations. We also propose negative content guidance, inspired by classifier-free guidance, to reduce content leakage in stylized image generation.

0.11

Salvation Lies Within: Eliciting Inherent Style Transfer in Step-Distilled Diffusion Models

cs.CV ·2026-10-04 ·Shengyin Sun, Yiming Li, Yingzhao Lian, Xing Li, Xingzhi Zhou +5 more
code/data similarity 0.65
Abstract · upload the wrong version

Adapting step-distilled text-to-image (T2I) models through post-training incurs additional computational costs and affects native few-step generation behavior. This motivates a complementary route beyond style-specific adaptation: drawing on the visual knowledge already encoded in step-distilled T2I models to elicit stylistic capabilities through language. Pursuing this direction requires textual guidance that captures how visual attributes jointly define a style and remain applicable as the depicted content changes. To explore this approach, we introduce StyleForge, a fully automatic, training-free framework that expresses reference styles as reusable rendering instructions. By integrating overall rendering characteristics with local color and lighting behavior, StyleForge organizes visual evidence from reference images into a coherent specification of how the target style should be expressed. The specification is then compiled into textual guidance that can be reused across content prompts, enabling frozen step-distilled T2I models to render different subjects and scenes in the reference style while retaining native few-step generation. Extensive experiments show relative gains of up to 29.47\% in generation quality scores over the strongest baseline, while Pareto analysis indicates that improved stylization is accompanied by strong adherence to the requested content. Code is available at https://github.com/sunshy-1/StyleForge.

0.35