CS Paper Radar

Handwriting Trajectory Recovery via Autoregressive Ordered Stroke Instance Prediction

En-Guang Wang, Yan-Ming Zhang, Fei Yin, Cheng-Lin Liu

cs.CV ·submitted 2026-09-02 07:57 UTC · v1
position

Abstract

Handwriting trajectory recovery aims to infer the dynamic writing process hidden behind a static handwritten image. Since offline handwriting preserves only the final spatial ink pattern, temporal information such as stroke order, writing direction, and pen-tip motion is lost, making recovery inherently ambiguous. Existing learning-based methods often directly predict the complete character trajectory without explicitly exploiting the stroke-level organization of handwriting. We argue that recovering the writing process should follow the writing process itself. Accordingly, we propose a two-stage framework that first recovers ordered stroke instances and then reconstructs continuous within-stroke motion. The first stage integrates stroke extraction and stroke-order recovery through autoregressive ordered stroke prediction, while direction-related structural cues further support within-stroke trajectory generation. Experiments on Chinese handwriting show that the proposed ordered prediction is more effective than post-hoc stroke ordering. Even without trajectory simplification, our full-point model achieves numerically better results than those reported by all compared baselines, while a controlled analysis shows that trajectory sampling density substantially affects measured recovery performance. Additional experiments demonstrate generalization to unseen Chinese character categories and cross-language extensibility to English and Tamil handwriting.

Comment
24 pages, 6 figures, 7 tables

Closest papers

Nearest neighbours by meaning across the whole index.

inkGround: Benchmarking Fine-grained Visual Grounding Capabilities of Vision-Language Models for Cursive Handwriting in the Wild

cs.CV cs.AI ·2026-10-28 ·Sidra Hanif
benchmarkcode/data similarity 0.75
Abstract

Human handwriting is produced as an ordered sequence of pen movements, but an offline image keeps only the final ink. Handwriting trajectory recovery aims to restore the ordered stroke points and their pen-up/pen-down transitions. Existing image-trajectory pair datasets are recorded with digitizing tablets under controlled conditions, which limits writer coverage and stylistic diversity. Moreover, there is no benchmark that measures the visual grounding capabilities of general-purpose Vision-Language Models (VLMs) for the stroke trajectory recovery task. In this work, we introduce inkGround, a benchmark for stroke trajectory recovery from full-page, camera-captured handwriting. inkGround pairs 537 page images (~18.7k words, ~163k strokes) with reference trajectories produced by a pretrained offline-to-online model, filtered by OCR agreement, and verified by human raters. Because these references are pseudo-labels that represent one plausible pen trajectory, we also evaluate VLMs on the IAM-OnDB dataset, which provides recorded pen trajectories. We cast the task as text-conditioned point grounding: given a page image and a target word, the model must locate the word and output its ordered stroke points. Outputs are scored by trajectory fidelity (DTW, Chamfer distance), readability of the rendered strokes (Accurate Rate, Correct Rate), and style similarity. We benchmark four open-source and three frontier closed-source VLMs with a shared in-context prompt. Closed-source models, notably Gemini-3.1-Pro, produce legible trajectories close to the reference, and this ranking also holds on IAM-OnDB.

0.26

Grip, Trace, Write: Force-Aware Imitation and Reinforcement Learning for High-DoF Robotic Handwriting

cs.RO ·2026-11-07 ·Rongqian Chen, Yiding Chen, Jijie Zhou, Tom Z. Jiahao
similarity 0.64
Abstract · This paper is accepted by CoRL 2026 Workshop PhysWM

Most robotic handwriting rigidly mounts the pen and reduces writing to trajectory tracking. A freely held pen instead couples grasp stability, tip motion, and contact force. We present Grip, Trace, Write, a force-aware simulation environment and learning framework for high-DoF handwriting. A two-branch actor separates pen retention from arm-wrist writing while preserving their closed-loop coupling. Training learns a grasp controller, initializes writing from inverse-kinematics demonstrations by behavior cloning, and improves writing through reinforcement learning with rewards on ink, force, time, and retention. Across diverse tasks, the policy improves completion, precision, force regulation, and efficiency over behavior cloning and inverse kinematics, and generalizes without fine-tuning to unseen English and Chinese content. These results establish force-aware handwriting as a testbed for dexterous tool use.

0.15

From Restoration to Redrawing: Revitalizing Ancient Chinese Characters by SVG Code Generation and Editing

cs.CV ·2026-11-18 ·Weiguang Zhang, Linhua Dong, Bin Dong, Shanshan Jiang, Maizhen Ning +7 more
similarity 0.63
Abstract

Hanjian manuscripts written on Han Dynasty bamboo and wooden slips are invaluable heritage of ancient Chinese writing but suffer from damaged ink and incomplete strokes. Existing image-to-image restoration methods may alter observed strokes and offer limited control over stroke geometry. Inspired by experts' manual redrawing practices, we address these issues by introducing \textbf{ReDraw}, a two-stage vision-language model(VLM) that reformulates character restoration as Scalable Vector Graphics(SVG) code generation and editing. Stage one targets faithful vectorization of observed strokes geometry or conservative tracing of mildly degraded strokes. Guided by skeleton chain-of-thought reasoning and reinforcement learning with structure-aware rewards, stage two predicts skeletons before contours to redraw severely incomplete strokes by appending paths while keeping existing SVG paths unchanged. To support training, we develop a multi-layer data synthesis pipeline that disentangles ink, material, and degradation to generate 1.2M samples, and derive up to 6,400 adaptation pairs from 320 annotated real glyphs. Experiments demonstrate synthetic restoration performance and adaptability to real Hanjian glyphs. Code and demo will be available upon acceptance.

0.29

Age-Adaptive Handwriting Reconstruction from an IMU-Based Digital Pen through Shared Representations and Domain-Specific Heads

cs.CL cs.AIcs.CVcs.LG+1 ·2026-09-15 ·Florent Imbert, Yann Soullard, Eric Anquetil, Hui Han
published similarity 0.62
Abstract

Digital pens are widely used to capture handwriting on digital devices, enabling precise trace recording and enhancing human-computer interaction. However, most are bundled with tablets and lack cross-brand compatibility. Recent digital pens equipped with kinematic sensors have emerged, designed for use on any surface. This especially opens significant potential for supporting handwriting acquisition in classrooms. Handwriting reconstruction from such an IMU-equipped pen poses a challenge due to the significant variability in sensor signals between adults and children. Even when producing visually similar traces, variations in writing dynamics, motor control, pen holding, and user confidence introduce substantial discrepancies in the captured signals. Additionally, the high variability in children's handwriting requires collecting large amounts of data, which is not feasible to implement in a school environment at scale. Furthermore, models trained exclusively on adult data fail to generalize to children's handwriting, and conversely, models trained on children's data perform poorly on adult writers. This crosspopulation degradation highlights the need for a unified model that can be deployed directly on the pen, without any user-specific adaptation. To address this issue, we propose a cross-domain learning, using an original neural network architecture based on a Temporal Convolutional Network and multiple prediction heads. The model is designed to be robust across age groups by leveraging shared features while effectively handling variability induced by differences in graphomotor development. This approach aims to improve handwriting trace reconstruction from sensor data, where each domain benefits from additional data provided by the other domain.

0.35

Segment-Level Risk Discovery in Online Handwriting for Alzheimer's Disease Detection

cs.CV cs.AI ·2026-09-24 ·Changqing Gong, Huafeng Qin, Mounîm A. El-Yacoubi
benchmark similarity 0.61
Abstract

Online handwriting provides a non-invasive and low-cost behavioral biomarker for Alzheimer's disease (AD) detection, as it reflects both cognitive planning and fine motor control. Existing handwriting-based AD detection methods usually rely on global trajectory features or whole-sample representations, which can be strongly affected by individual writing style, task-specific variation, and acquisition noise. In this paper, we propose NormPaST-Risk, a healthy-normative Paper-Air selective trajectory state-space risk network for interpretable AD detection from online handwriting. Instead of treating the entire trajectory as a single holistic representation, our method reformulates AD handwriting detection as local disease-relevant segment discovery. Specifically, a multi-scale temporal encoder captures stroke dynamics at different temporal resolutions, while a selective Paper-Air state-space encoder models long-range handwriting progression and distinguishes on-paper motor execution from in-air planning and transition behaviors. To explicitly characterize abnormal deviations, a healthy normative branch learns normal handwriting dynamics from healthy controls, and a task-aware multi-expert segment-risk module estimates segment-level AD risk calibrated by hidden-state changes and normative deviations. A weakly supervised segment-level objective further enables high-risk segment discovery without manual segment annotations. Experiments on the DARWIN benchmark demonstrate that the proposed framework achieves superior AD/HC classification performance compared with existing methods. Moreover, the discovered high-risk segments can be projected back to the original handwriting trajectory, providing interpretable evidence associated with AD-related handwriting variations.

0.24

Japanese Handwriting Generation Based on Human Preference Feedback Using Direct Preference Optimization

cs.CV ·2026-10-10 ·Wakana Kuwata, Ryota Mibayashi, Masanori Tani, Hiroaki Ohshima
published similarity 0.60
Abstract · Accepted for publication at BESC 2026

We propose a Japanese handwriting generation framework that integrates human preference feedback using Direct Preference Optimization (DPO) to align generative models with human cognitive and stylistic preferences. While font generation technologies leveraging Generative Adversarial Networks and diffusion models have advanced significantly, applying these approaches to handwriting generation presents unique challenges. Font generation is a deterministic process yielding consistent styles. Handwriting generation, on the other hand, is inherently non-deterministic. In handwriting, glyph shapes never match exactly even when the same writer writes the same character. We define this shape difference as handwriting variability. Although this variability is essential for conveying the natural quality of human writing, it complicates model training and substantially increases data collection costs to capture diverse distribution patterns. To address these limitations, we incorporate human perceptual feedback into the training loop of handwriting generation. While collecting extensive handwritten data is costly, evaluating pairwise preferences between two generated samples remains very easy for humans. Applying DPO, an efficient RL-free preference optimization method, to our generation model enables high-quality handwriting generation without relying on large-scale dataset collection. Experimental results across four distinct Japanese handwriting styles demonstrate that DPO-based training enhances overall structural accuracy. However, in highly complex styles with significant variability, such as brush pen writing, the model fails to capture style-specific features. These findings highlight a critical challenge in extending DPO to complex handwriting styles.

0.30

Handwritten Text Recognition Lives in the High-Pixel Variance Subspace

cs.CV cs.LG ·2026-09-28 ·Carlos Garrido-Munoz, Jorge Calvo-Zaragoza
top venuebenchmarkposition similarity 0.59
Abstract · Accepted at 40th Conference on Neural Information Processing Systems (NeurIPS 2026)

In self-supervised pretraining for Handwritten Text Recognition (HTR), pixel reconstruction methods outperform contrastive methods, unlike in natural-image classification. We argue that this difference follows from where discriminative signal lies in pixel space: for HTR, it is concentrated in high-variance directions and largely absent from low-variance ones. This predicts that objectives preserving high-variance pixel content will transfer best. We test six SSL methods from three families (pixel-grounded MIM, JEPA, and contrastive) under matched encoder, data, and evaluation protocols on six handwriting benchmarks across five languages. With full labels, pixel-groundrounded SSL achieves the lowest CER on every benchmark and both frozen probes, exposes per-position character information that other families recover only through the readout, and is the only family to benefit from pretraining on real handwriting. Pixel-grounded representations are also more label efficient. Across datasets, encoder alignment with the high-variance pixel subspace predicts CER within every method. With a pretrained LLM decoder, a frozen pixel-grounded encoder is competitive with fully fine-tuned supervised baselines; full fine-tuning achieves the lowest mean CER and ranks first or second on every benchmark. These results show that the value of pixel reconstruction depends on where discriminative signal lies in the input.

0.38

Write on Paper and Get the Online Digital Trace: A New Era for Handwriting

cs.LG ·2026-09-11 ·Florent Imbert, Yann Soullard, Eric Anquetil, Tanja Harbaum, Alexey Serdyuk +3 more
published similarity 0.58
Abstract

Capturing the digital trace of handwriting usually requires a specific stylus and a compatible substrate, be it a capacitive touchscreen, an ElectroMagnetic Resonance (EMR) tablet as used in Wacom systems or special paper. While writing on regular paper offers rich haptics, no latency and is well known for improving information retention, no low-cost and widely accepted, effective solution exists to digitize such a pen trace. The challenge is to accurately track the pen's trajectory without an external reference system while allowing unrestricted freedom of pen movement across a surface. We propose an innovative solution that combines a digital pen, advanced artificial intelligence algorithms, and adaptive AI techniques to reconstruct the digital trace of handwriting. Our approach integrates hardware development, focusing on a sensor-equipped pen, with software innovations to optimize trajectory reconstruction and processing in real time using an embedded AI. This work aims to advance the state-of-the-art in automated trace reconstruction of handwriting, enabling a seamless connection between traditional handwriting on paper and capturing the trace digitally.

0.65