CS Paper Radar

Japanese Handwriting Generation Based on Human Preference Feedback Using Direct Preference Optimization

Wakana Kuwata, Ryota Mibayashi, Masanori Tani, Hiroaki Ohshima

cs.CV ·submitted 2026-10-10 09:03 UTC · v1
published

Abstract

We propose a Japanese handwriting generation framework that integrates human preference feedback using Direct Preference Optimization (DPO) to align generative models with human cognitive and stylistic preferences. While font generation technologies leveraging Generative Adversarial Networks and diffusion models have advanced significantly, applying these approaches to handwriting generation presents unique challenges. Font generation is a deterministic process yielding consistent styles. Handwriting generation, on the other hand, is inherently non-deterministic. In handwriting, glyph shapes never match exactly even when the same writer writes the same character. We define this shape difference as handwriting variability. Although this variability is essential for conveying the natural quality of human writing, it complicates model training and substantially increases data collection costs to capture diverse distribution patterns. To address these limitations, we incorporate human perceptual feedback into the training loop of handwriting generation. While collecting extensive handwritten data is costly, evaluating pairwise preferences between two generated samples remains very easy for humans. Applying DPO, an efficient RL-free preference optimization method, to our generation model enables high-quality handwriting generation without relying on large-scale dataset collection. Experimental results across four distinct Japanese handwriting styles demonstrate that DPO-based training enhances overall structural accuracy. However, in highly complex styles with significant variability, such as brush pen writing, the model fails to capture style-specific features. These findings highlight a critical challenge in extending DPO to complex handwriting styles.

Comment
Accepted for publication at BESC 2026

Closest papers

Nearest neighbours by meaning across the whole index.

From Generation to Matching: A Development Report on Personalized Chinese Handwriting

cs.CV ·2026-08-16 ·Yiwei Liu
similarity 0.64
Abstract

This paper documents a frozen engineering project on personalized Chinese handwriting. The project started from approximately 200 real handwriting images from one user, covering 197 unique Chinese characters, and was initially formulated as few-shot generation of unseen characters. A sequence of canonical-centered personalization routes repeatedly exposed the same conflict: increasing structural pressure made outputs more canonical, while increasing personalization could damage identity-defining strokes. The project was therefore reset around real-human character equivalence classes. A multi-writer CASIA candidate pool showed that a USER-compatible realization often already existed among valid human samples. The task consequently changed from synthesis to character-wise matching, followed by cross-writer composition into a virtual writer. The frozen system uses real-ink features, character-specific human population percentiles, top-20 candidate pruning, and greedy hardest-first whole-row selection. On the covered target set, all 197 USER characters had real-human candidates, and the 100-character evaluation subset was covered 100/100. Knowncharacter held-out comparisons included a row judged visually almost indistinguishable from genuine USER handwriting. A 60- episode stability audit placed every episode in a predefined A-like machine-proxy region, but these were not independent human A-level judgments. The final evidence supports stable practical B-level quality, with many outputs approaching A-level under the USER-defined criterion. The report records why generation became unnecessary for this case without claiming unrestricted or universal handwriting synthesis.

0.04

LoGAN: Multilingual Font Localization with Generative Agents

cs.CV cs.AI ·2026-09-07 ·Zhuoning Yuan, Ta-Ying Cheng, Benjamin Klein
top venue similarity 0.62
Abstract · Accepted by ECCV2026

Localizing a font into new languages is a highly intricate task requiring precise design adaptation of glyphs, color/texture, and spacing/kerning, from source to target languages. Most existing methods focus on single glyph generation with limited capability in handling multilingual font rendering. In this work, we propose LoGAN, a VLM-based agentic framework for few-shot multilingual font localization, which takes in a small number of individual glyphs from a font or letters from a logo and uses them to generate complete character sets in other languages. LoGAN breaks down this task into multiple components: a glyph-level diffusion model, a style finetuning module, a spacing and kerning transfer algorithm, and a texture expansion model, with a VLM agent coordinator. LoGAN achieves broad language coverage for font localization with various styles, including Chinese/Korean/Japanese (CJK). We evaluate our approach on both font and real-world logo datasets spanning more than 27 languages and compare it against both specialized font generation and state-of-the-art image editing models with strong text rendering capabilities (e.g., FLUX, Nano-Banana). Our approach yields higher glyph fidelity while maintaining better style, texture, and kerning consistency according to both quantitative and qualitative evaluations.

0.43

Disentangled Stylized Image Generation via IP-Cross-Attention Rearrangement

cs.CV ·2026-11-07 ·Beibei Dong, Yueming Lyu, Yue Jiang, Jing Dong
similarity 0.61
Abstract · 12 pages, 11 figures

With a solid theoretical foundation and pretraining on large-scale datasets, diffusion-based methods have achieved controllable generation leveraging text and image conditions, and generating excellent stylized images is possible using diffusion models. However, previous methods face significant challenges. First, because diffusion generation is iterative, the common approach of learning implicit style representations via cycle-consistency strategies is too costly and not well-suited to diffusion, even for LoRA-based methods. Second, decoupling style is difficult in the latent space of diffusion models. Representations of attributes are coupled in the latent space of diffusion models; thus, extracting style attributes with diffusion models usually suffers from the issue of content leakage. Furthermore, because style is hard to represent in natural language, capturing style using text prompts or textual inversion is deficient. In this paper, we propose a method for achieving disentangled stylized image generation through IP-cross-attention rearrangement, based on our experimental observations. We also propose negative content guidance, inspired by classifier-free guidance, to reduce content leakage in stylized image generation.

0.11

From Restoration to Redrawing: Revitalizing Ancient Chinese Characters by SVG Code Generation and Editing

cs.CV ·2026-11-18 ·Weiguang Zhang, Linhua Dong, Bin Dong, Shanshan Jiang, Maizhen Ning +7 more
similarity 0.60
Abstract

Hanjian manuscripts written on Han Dynasty bamboo and wooden slips are invaluable heritage of ancient Chinese writing but suffer from damaged ink and incomplete strokes. Existing image-to-image restoration methods may alter observed strokes and offer limited control over stroke geometry. Inspired by experts' manual redrawing practices, we address these issues by introducing \textbf{ReDraw}, a two-stage vision-language model(VLM) that reformulates character restoration as Scalable Vector Graphics(SVG) code generation and editing. Stage one targets faithful vectorization of observed strokes geometry or conservative tracing of mildly degraded strokes. Guided by skeleton chain-of-thought reasoning and reinforcement learning with structure-aware rewards, stage two predicts skeletons before contours to redraw severely incomplete strokes by appending paths while keeping existing SVG paths unchanged. To support training, we develop a multi-layer data synthesis pipeline that disentangles ink, material, and degradation to generate 1.2M samples, and derive up to 6,400 adaptation pairs from 320 annotated real glyphs. Experiments demonstrate synthetic restoration performance and adaptability to real Hanjian glyphs. Code and demo will be available upon acceptance.

0.29

Handwriting Trajectory Recovery via Autoregressive Ordered Stroke Instance Prediction

cs.CV ·2026-09-02 ·En-Guang Wang, Yan-Ming Zhang, Fei Yin, Cheng-Lin Liu
position similarity 0.60
Abstract · 24 pages, 6 figures, 7 tables

Handwriting trajectory recovery aims to infer the dynamic writing process hidden behind a static handwritten image. Since offline handwriting preserves only the final spatial ink pattern, temporal information such as stroke order, writing direction, and pen-tip motion is lost, making recovery inherently ambiguous. Existing learning-based methods often directly predict the complete character trajectory without explicitly exploiting the stroke-level organization of handwriting. We argue that recovering the writing process should follow the writing process itself. Accordingly, we propose a two-stage framework that first recovers ordered stroke instances and then reconstructs continuous within-stroke motion. The first stage integrates stroke extraction and stroke-order recovery through autoregressive ordered stroke prediction, while direction-related structural cues further support within-stroke trajectory generation. Experiments on Chinese handwriting show that the proposed ordered prediction is more effective than post-hoc stroke ordering. Even without trajectory simplification, our full-point model achieves numerically better results than those reported by all compared baselines, while a controlled analysis shows that trajectory sampling density substantially affects measured recovery performance. Additional experiments demonstrate generalization to unseen Chinese character categories and cross-language extensibility to English and Tamil handwriting.

0.41

Data-Driven Glyph Positioning for Handwritten Music Synthesis

cs.CV ·2026-10-20 ·Elona Shatri, Jiří Mayer, Jan Hajič
published similarity 0.59
Abstract · 10 pages, 1 biblio page, presented at The 20th International Conference on Document Analysis and Recognition

Synthesised data is a common strategy to address the lack of annotated handwritten music scores in Optical Music Recognition (OMR). Smashcima is a recent synthesis framework that assembles pages of handwritten scores by sampling glyph images from MUSCIMA++ and positioning them with hard-coded rules. In this work we ask whether these rules are grounded empirically and whether a learned model for context-dependent positioning can further improve synthesis quality. We extract inter-glyph distributions from around 285 thousand annotated relationships across MUSCIMA++ and OmniOMR, finding that several key parameters deviate systematically from real handwriting. Replacing these constants with empirical means reduces positioning RMSE by up to 60%. We then train a Graph Neural Network (GNN) on the Music Notation Graph structure to predict context-dependent offsets, achieving an additional 50 to 85% RMSE reduction on stems and beams by modelling how neighbouring glyphs constrain placement. Cross-dataset analysis shows that spacing differences between the two corpora are explained by note density (3.3), not style, and that duration-proportional spacing does not hold in handwriting (R2 = 0.001). In downstream evaluation, a YOLOv8 detector pretrained on GNN-positioned synthetic data achieves +30% mAP50 over the baseline and +11% over empirical means when fine-tuned on 200 real handwritten staves, showing that positioning accuracy matters for downstream OMR.

0.35

T-DIG: Text-Conditioned Diversity for Image Generation

cs.CV ·2026-10-21 ·Simran Khanuja, Rohit Gandikota, Gaurav Parmar, Graham Neubig
similarity 0.58
Abstract · under review

Text-to-image (T2I) platforms typically return several images per prompt so users can explore different visual interpretations and choose one that matches their intent. However, T2I models are usually optimized for single-image quality which comes at a cost of multi-image diversity, limiting this exploration. Diversity is also contextual: one user wants variations in background, while another wants them in artistic style, and the same user may later want variations in clothing. The generated images should differ along whatever axis the user cares about. In this paper, we introduce T-DIG, a text-conditioned diversity framework for image generation and editing. T-DIG is an iterative framework comprising multiple modules that lets users type prompts, inspect default generations, request variations using natural language, and continue refining outputs until they're satisfied. At its core, T-DIG uses an LLM policy that produces K axis-conditioned prompts or edit instructions for a frozen generator or editor. We train this policy with GRPO using a text-conditioned diversity reward and alignment penalties. Because the image generator remains frozen, T-DIG applies to open-weight, distilled, and proprietary models. We also release the first dataset for text-conditioned prompt diversity in image generation, with the training and validation splits built from human-written axes and a test set drawn from real user interactions with T-DIG. T-DIG far exceeds baselines in improving text-conditioned diversity, while preserving base-prompt alignment across a variety of metrics and human judgements. We show that user-directed diversity exploration can be added to a wide range of image generation models, open and closed, without the need to access their weights. Website: https://image-study.org/t-dig/

0.41

Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards

cs.CV cs.AI ·2026-10-02 ·Yuanhao Ban, I-Hung Hsu, Anastasios Angelopoulos, Wei-Lin Chiang, Ion Stoica +1 more
benchmarkcode/data similarity 0.58
Abstract

Recent text-to-image generation models have achieved remarkable visual quality, but improving them through post-training remains challenging because no single reward signal captures the full range of human preference. In this work, we develop a simple and effective post-training recipe for open-domain text-to-image generation based on the composition of complementary reward signals. Our reward system consists of two main components: a preference reward, trained on large-scale human preference data using a Bradley-Terry objective to capture overall human aesthetic and perceptual preferences, and rubric-based rewards, which explicitly evaluate prompt faithfulness and other desirable properties while providing safeguards against reward hacking. A key challenge is how to combine these heterogeneous reward signals. We show that a naive weighted average leads to suboptimal optimization behavior, and propose a simple reward composition strategy that more effectively balances preference optimization with rubric satisfaction. In the Arena text-to-image leaderboard (https://arena.ai/), our RL-trained Flux2dev achieves an Elo rating 69 points above the base model, and our post-trained Ideogram-4 surpasses every open-source model on the leaderboard, reaching an Elo of 1223.5. (Claims of state-of-the-art performance are based on the Arena leaderboard snapshot as of September 4, 2026.) Our results suggest that effective rewards for frontier generative-model training require broad coverage of user intent and robustness to exploitation under optimization. To support reproducible research, we release Arena-T2I-Training, a 1K subset of training data that recovers some gains of full-scale training, providing a resource that we hope will facilitate future work on post-training for text-to-image models.

0.31