CS Paper Radar

Measuring Annotation Efficiency for Handwritten Devanagari Recognition: Sample-Complexity Curves for Four Pretraining Regimes

Manglesh Kumar Pandey, Sumit Kumar Banshal

cs.CV cs.LG ·submitted 2026-09-15 08:50 UTC · v1

Abstract

To train handwritten text recognition systems we need word images and their corresponding transcriptions, and these transcriptions are produced manually. For a script that can be read by only a small number of specialists, this manual transcription is a limitation, because the trained models are supposed to save the time of those same specialists. A relevant question therefore arises: how many transcriptions are needed before a recogniser becomes useful, and how much of that cost can pretraining remove? In this study the answer is measured directly for handwritten Devanagari. We keep the recogniser, optimiser and evaluation protocol the same and change only the number of real transcribed words used for fine-tuning across nine budgets from 10 to 4,000 and four initialisation regimes, with six seeds at every point. The resulting curves are then converted into annotation-equivalent terms. A CER of 0.50 is reached by supervised synthetic pretraining using only 81 transcribed words, whereas random initialisation requires 355, which gives a label multiplier of 4.40 [3.56, 4.99]. There is a zero-shot reference point as well: with no real transcribed words at all, this pretraining is worth about 136 of them. This advantage gets smaller as the target accuracy improves, and at the most demanding target we measure, it cannot be distinguished from no saving at all. A fourth arm in which only the encoder is transferred separates the effect of the pretraining method from that of transfer scope, and masked image modelling is observed to transfer negatively over a bounded range of budgets. We emphasise that the scarcity in this study is constructed by subsampling a large corpus.

Closest papers

Nearest neighbours by meaning across the whole index.

OmniHandwritingOCR: A Diagnostic Benchmark for Evaluating Multimodal LLMs in Handwritten OCR Scenarios

cs.CV cs.AI ·2026-08-19 ·Zinuo Guo, Min Zhang, Bo Jiang
top venuebenchmarkhot direction similarity 0.68
Abstract · CIKM 2026

Multimodal large language models (MLLMs) are increasingly used as OCR systems in document and knowledge-processing pipelines, but their ability to faithfully read real handwriting remains underexplored. Existing OCR benchmarks focus largely on printed text or clean single-line inputs, leaving limited coverage of realistic handwritten OCR scenarios such as multilingual handwriting, writer errors, and structurally complex mathematical expressions. We introduce OmniHandwritingOCR, a diagnostic benchmark for evaluating MLLMs and OCR systems on handwritten OCR. It covers handwritten text recognition and handwritten mathematical expression recognition across six subtasks and twelve subsets, totaling 77.57K labeled images from public datasets and newly collected student writings. A key component is a difficulty-stratified multi-line formula corpus designed to test robustness under increasing structural complexity. We evaluate thirteen open- and closed-source systems with five complementary metrics under a unified protocol. Results show that current systems remain far from faithful transcription: performance drops sharply on complex multi-line formulas, model rankings vary across language and formula settings, and several generative models hallucinate plausible but visually unsupported corrections. OmniHandwritingOCR provides a challenging testbed for diagnosing language, content, structural, and visual-grounding failure modes of multimodal models in handwritten OCR scenarios.

0.51

Frugal Handwritten Text Recognition for Sub-Saharan Civil-Status Registers: A Smartphone-to-Records Pipeline with CPU-Only Inference

cs.CV ·2026-10-27 ·Ousmane Hamadou, Jean Louis Kedieng Ebongue Fendji
published similarity 0.65
Abstract · Accepted at the GlobalSouthCV Workshop, BMVC 2026. 17 pages, 3 figures

Handwritten civil-status registers across Sub-Saharan Africa are being photographed, but rarely transcribed, which leaves their content unsearchable. Local administrations that hold these records typically lack GPUs, reliable connectivity, and the budget for external digitization services. We study whether a frugal pipeline, with consumer-grade capture and CPU-only inference, can make such registers machine-readable. We introduce a corpus of 1,568 smartphone images of handwritten birth, marriage, and death records from a municipal registry in Chad, with line-level transcriptions for 185 records. Our pipeline combines a fully classical preprocessing stage, running at about 2.6 images per second on a laptop CPU, with a Kraken recognizer fine-tuned from a pretrained model in about four hours on a free-tier GPU. On manually corrected line segmentation and a single-registry test set whose writers also appear in training, fine-tuning reduces the character error rate from 14.53% to 0.50% and the word error rate from 56.93% to 3.37%. On a six-core laptop CPU, the recognizer weighs 61 MiB, peaks below 2.6 GiB of RAM, and processes an image in under 6 s on average, excluding line segmentation. Finally, a downstream information-extraction task shows that residual recognition errors cost about 9 F1 points on role resolution, compared with a 36.7-point gap between the fine-tuned and pretrained recognizers. These results are evidence of feasibility within one registry, not of performance on unseen writers or sites; we discuss these limitations in detail.

0.37

Exploring In-Context Learning for Handwritten Text Recognition

cs.CV ·2026-09-29 ·Eric Ayllon, Abel Gandia, Jorge Calvo-Zaragoza
similarity 0.64
Abstract · 19 pages, 3 figures

Handwritten Text Recognition (HTR) systems have become an indispensable tool for the digitization of historical documents. Not only do they cut down time and cost, but they also allow democratizing access and processing of their contents by generating their transcripts. However, literature in HTR currently focuses mostly on specialized models that require large amounts of annotated samples to achieve satisfactory performance. We explore the use of In-Context Learning with pre-trained Vision-Language Models (VLMs) to create a transcription pipeline without updating the model's parameters. We then evaluate this pipeline across multiple collections and models, and demonstrate that general-purpose VLMs can be effectively taught how to transcribe handwritten text from images. To assess how our observations may translate to practical applications, we evaluate the performance in a Cross-Domain (CD) scenario, where context examples are drawn from a different collection than the query image. Results in both the controlled In-Domain (ID) scenario and the realistic CD scenario follow the same patterns. First, as context size grows, the error range is expected to narrow towards the average performance. Thus, larger context sizes sacrifice the performance of the oracle-best sampling for lower expected error rates. The results obtained show that, without any parameter updates, this methodology has strong potential to compete with traditional HTR in the presence of domain shift. Moreover, we show and argue that some context samplings work better than others and suggest more effort should be put into finding an ideal sampling method in future work.

0.30

Dataset Distillation for Handwritten Text Recognition: An Empirical Study

cs.CV ·2026-11-16 ·Félix Fuentes-Hurtado, Aric Ayllón, Juan C. Alfaro
similarity 0.63
Abstract

Dataset distillation (DD) replaces large training datasets with smaller synthetic ones to reduce computational costs. While traditionally used for classification, we present, to our knowledge, the first DD study for line-level handwritten text recognition (HTR). Unlike classification tasks, which use discrete labels, HTR employs unaligned sequences from a virtually infinite output space as labels. To adapt classification-based DD methods to this scenario, we define text pseudo-classes by clustering transcriptions based on character n-grams. We adapt gradient matching (GM), distribution matching (DM) and trajectory matching (MTT) to a Convolutional Recurrent Neural Network (CRNN) trained with Connectionist Temporal Classification (CTC). In this process, synthetic line images are generated by optimising only the pixels of real starting lines (either randomly chosen or selected as text medoids), while keeping their original transcriptions fixed. Across four datasets (IAM, George Washington, READ\,2016, LAM) and three budgets (1--10\,\% of the training lines), no distilled set outperforms its real initialisation lines. Random lines prove better than medoids, and further optimisation only degrades recognition. Methodologically, DM lowers its objective without benefit, while GM and MTT barely move theirs. We turn these findings into guidelines for evaluating data compression in HTR.

0.18

Devanagari Handwritten Character Recognition Using TrOCR: A Transformer-Based Model with Real-Time Web Deployment

cs.CV ·2026-09-16 ·Amrit Baskota, Samyam Budhathoki, Shubham Ghimire, Abiskar Ghimire, Sarwesh Phuyal +1 more
published similarity 0.62
Abstract · Presented and accepted at the 16th International Conference on Computing, Communication and Networking Technologies (ICCCNT), 2025

Devnagari is a one of the ancient language of the Indian subcontinent consisting of 36 vowels, 14 consonants and 10 numerals. The accurate recognition of handwritten Devnagari characters is challenging due to high complexity of Devnagari scripts. This paper presents a method to fine tune the pre trained TrOCR model to accurately recognize Devanagari handwritten characters. The Methodology consists of a preprocessing mechanism where input images are standardized into RGB format, tokenized in batches and integrated with Hugging Face Dataset. The pre-trained microsoft/trocr-base-handwritten model is fine-tuned over a dataset of nearly 5000 character images that are uniformly partitioned in the ratio 8:1:1 for training, evaluation and testing. Further optimization is done is the training process through mixed precision training, gradient checkpointing, and early stopping mechanism. The model achieves a character error rate (CER) of 3.95% and a character-level accuracy of 96.05%, outperforming the previous CNN based models. A scalable web application is developed using Vite.js, Golang, and FastAPI which practically deploys the OCR model and serves character recognition task with a latency less than 5 seconds per request. This study demonstrates the use of TrOCR model to build a scalable handwritten Devnagari character recognition system and also a foundation to future research on Devnagari Script Recognition using Transformers.

0.41

Handwritten Text Recognition Lives in the High-Pixel Variance Subspace

cs.CV cs.LG ·2026-09-28 ·Carlos Garrido-Munoz, Jorge Calvo-Zaragoza
top venuebenchmarkposition similarity 0.61
Abstract · Accepted at 40th Conference on Neural Information Processing Systems (NeurIPS 2026)

In self-supervised pretraining for Handwritten Text Recognition (HTR), pixel reconstruction methods outperform contrastive methods, unlike in natural-image classification. We argue that this difference follows from where discriminative signal lies in pixel space: for HTR, it is concentrated in high-variance directions and largely absent from low-variance ones. This predicts that objectives preserving high-variance pixel content will transfer best. We test six SSL methods from three families (pixel-grounded MIM, JEPA, and contrastive) under matched encoder, data, and evaluation protocols on six handwriting benchmarks across five languages. With full labels, pixel-groundrounded SSL achieves the lowest CER on every benchmark and both frozen probes, exposes per-position character information that other families recover only through the readout, and is the only family to benefit from pretraining on real handwriting. Pixel-grounded representations are also more label efficient. Across datasets, encoder alignment with the high-variance pixel subspace predicts CER within every method. With a pretrained LLM decoder, a frozen pixel-grounded encoder is competitive with fully fine-tuned supervised baselines; full fine-tuning achieves the lowest mean CER and ranks first or second on every benchmark. These results show that the value of pixel reconstruction depends on where discriminative signal lies in the input.

0.38

UniLipi: A Unified Multi-Script OCR for Historical Indic Manuscripts

cs.CV ·2026-08-28 ·Tathagata Ghosh, Sai Madhusudan Gunda, Simran Singh Sandral, Ravi Kiran Sarvadevabhatla
top venue similarity 0.60
Abstract · To be published in ICDAR 2026

Optical character recognition (OCR) for handwritten Indic manuscripts is essential for large-scale digitization and computational access to manuscript heritage. However, existing approaches are typically developed for one script at a time and require substantial script-specific customization. This limits scalability and practical deployment across diverse collections. We present UniLipi, a unified multi-script OCR model for handwritten Indic manuscripts trained jointly across 13 Indic scripts within a single framework. UniLipi directly handles realistic manuscript conditions, including extreme variation in line geometry, large variation in line length, and partial interruptions caused by non-textual manuscript entities such as holes, stains, or pictorial illustrations. To operate effectively under ultra low-resource conditions, the model leverages script-aware synthetic manuscript data generation, substantially reducing reliance on large volumes of real annotated data. Beyond historical manuscripts, we show that UniLipi serves as an effective foundational pretrained model. Specifically, its learned representations enable good OCR performance for contemporary Indic handwriting and extend to several non-Indic scripts, including Tibetan, Italian, Latin, and Chinese scripts. In addition to transcription, UniLipi predicts script identity and per-line native character counts, supporting practical manuscript cataloging workflows.

0.35

Confusion-Adaptive Margin Learning for Handwritten Letter Recognition: A Controlled Study on EMNIST Letters

cs.CV cs.LG ·2026-10-22 ·Azam Nouri
similarity 0.60
Abstract

Handwritten character recognition systems can achieve high accuracy while retaining concentrated errors among visually similar classes. Motivated by residual letter confusions in earlier geometry-based recognition work, we investigate whether these errors can be addressed through the training objective. We introduce Confusion-Adaptive Margin Learning (CAML), which constructs a sparse weighted confusion graph from a discovery partition held out from pilot training and applies pair-specific cosine-separation constraints during final training. CAML adds no classifier, routing rule, or inference-time parameters. We evaluate CAML on EMNIST Letters against a strictly matched cross-entropy CNN baseline, using the same architecture, final training data, deterministic augmentation, optimizer, and checkpoint selection. Eight confusion pairs are targeted: I/L, G/Q, U/V, R/V, D/O, C/E, I/J, and H/N. On the official clean test set, accuracy changes from 95.6346% to 95.6731%, and macro-F1 from 0.956326 to 0.956724. Mean accuracy across ten predefined corrupted conditions increases from approximately 95.3812% to 95.4303%; symmetric errors within targeted pairs decrease from 614 to 605 (1.47%). The global differences are small and not statistically reliable. CAML fixes 124 baseline errors but introduces 116 new errors on clean data (exact McNemar p=0.6515), and no corruption-specific difference remains significant after Holm correction. Thus, CAML does not establish global superiority, but shows that a leakage-controlled, pair-specific margin objective can alter targeted error structure with limited overall benefit for an already strong classifier.

0.19