CS Paper Radar

Full-Page Optical Music Recognition of Handwritten Monophonic Scores

Adrian Rosello, Antonio Ríos-Vila, David Rizo, Jorge Calvo-Zaragoza

cs.CV cs.AI ·submitted 2026-09-04 18:56 UTC · v1

Abstract

Full-page end-to-end Optical Music Recognition seeks to transcribe entire music pages directly into symbolic notation, avoiding the limitations of traditional pipelines that rely on accurate staff segmentation. Recent Transformer-based architectures have achieved strong performance on typeset scores, relying on large-scale synthetic data for pretraining. However, their applicability to handwritten music remains largely unexplored. In this work, we study full-page transcription on handwritten monophonic collections and analyze the impact of synthetic pretraining in this setting. To investigate which factors are most relevant during pretraining, we introduce a generator capable of producing visually coherent full-page scores in both typeset and handwritten styles. Experiments on three real handwritten datasets provide a comparative evaluation of several full-page pipelines and different synthetic pretraining strategies. The results suggest that the benefits of synthetic pretraining are primarily associated with learning structural layout conventions rather than with visual similarity to the target handwriting.

Closest papers

Nearest neighbours by meaning across the whole index.

Data-Driven Glyph Positioning for Handwritten Music Synthesis

cs.CV ·2026-10-20 ·Elona Shatri, Jiří Mayer, Jan Hajič
published similarity 0.66
Abstract · 10 pages, 1 biblio page, presented at The 20th International Conference on Document Analysis and Recognition

Synthesised data is a common strategy to address the lack of annotated handwritten music scores in Optical Music Recognition (OMR). Smashcima is a recent synthesis framework that assembles pages of handwritten scores by sampling glyph images from MUSCIMA++ and positioning them with hard-coded rules. In this work we ask whether these rules are grounded empirically and whether a learned model for context-dependent positioning can further improve synthesis quality. We extract inter-glyph distributions from around 285 thousand annotated relationships across MUSCIMA++ and OmniOMR, finding that several key parameters deviate systematically from real handwriting. Replacing these constants with empirical means reduces positioning RMSE by up to 60%. We then train a Graph Neural Network (GNN) on the Music Notation Graph structure to predict context-dependent offsets, achieving an additional 50 to 85% RMSE reduction on stems and beams by modelling how neighbouring glyphs constrain placement. Cross-dataset analysis shows that spacing differences between the two corpora are explained by note density (3.3), not style, and that duration-proportional spacing does not hold in handwriting (R2 = 0.001). In downstream evaluation, a YOLOv8 detector pretrained on GNN-positioned synthetic data achieves +30% mAP50 over the baseline and +11% over empirical means when fine-tuned on 200 real handwritten staves, showing that positioning accuracy matters for downstream OMR.

0.35

A Dataset and Benchmark for Optical Music Recognition of String Quartet Scores

cs.CV cs.SD ·2026-08-11 ·Dongmin Kim, Brian Liu, Jose J. Valero-Mas, Dasaem Jeong
publishedbenchmarkcode/data similarity 0.62
Abstract · 8 pages, 2 figures, 5 tables. Accepted at the ISMIR 2026

Optical music recognition (OMR) transcribes music scores into digital formats. While the field has advanced significantly on monophonic and piano-form scores, multi-part score transcription remains underexplored, largely due to the absence of a suitable dataset. We introduce OpenScore String Quartet for Optical Music Recognition (OSSQ-OMR), the first dataset dedicated to multi-part OMR. Built on the OpenScore String Quartet corpus, OSSQ-OMR pairs digitally encoded scores with their original scanned editions from IMSLP, with all images visually aligned to their transcriptions. The dataset is released with score images at system and staff levels, and paired transcriptions in three encoding formats: Extended Linearized MusicXML (LMXE), **kern, and ABC. In total, OSSQ-OMR contains 24,544 system images and 98,172 staff images drawn from 116 string quartet scores. We accompany the dataset with a benchmark protocol and baseline results from two representative OMR models, evaluated across four random score-level splits with mutually exclusive test sets. Baselines reach OMR-NED as low as 3.6% on synthetic and 5.9% on scanned inputs; results reveal substantial effects of encoding and segmentation choices, with the LSTM-based baseline degrading on scanned inputs roughly 2.6 times less than the Transformer-based baseline.

0.41

SheetSage2: Coherent Lead-Sheet Transcription with Synthetic Supervision

cs.SD ·2026-10-04 ·Junyan Jiang, Ruibin Yuan, Jiahao Pan, Wei Xue, Yike Guo +2 more
benchmarkcode/data similarity 0.62
Abstract

Transcribing music into a human-readable score requires a coherent understanding of rhythm, harmony, melody, and form. Two obstacles limit this goal: annotated recordings are scarce, and accurate local predictions can still produce inconsistent musical sequences. We present SheetSage2, a unified music transcription framework that combines synthetic data, task-specific structured decoding, and autoregressive distillation. Automatically annotated MIDI, rendered into audio, provides scalable supervision across music understanding tasks. Task-specific structured decoders integrate complementary musical cues and their temporal dependencies to produce musically coherent scores. Autoregressive distillation further retains transcription accuracy without task-specific dynamic programming at inference. Across eight benchmark collections, a single SheetSage2-AR model exceeds the listed prior systems on 12 of 15 benchmark--metric pairs in our evaluation, substantially improving over SheetSage1 and surpassing task-specific models on several benchmarks. Model weights and inference code are publicly available.

0.46

UniLipi: A Unified Multi-Script OCR for Historical Indic Manuscripts

cs.CV ·2026-08-28 ·Tathagata Ghosh, Sai Madhusudan Gunda, Simran Singh Sandral, Ravi Kiran Sarvadevabhatla
top venue similarity 0.59
Abstract · To be published in ICDAR 2026

Optical character recognition (OCR) for handwritten Indic manuscripts is essential for large-scale digitization and computational access to manuscript heritage. However, existing approaches are typically developed for one script at a time and require substantial script-specific customization. This limits scalability and practical deployment across diverse collections. We present UniLipi, a unified multi-script OCR model for handwritten Indic manuscripts trained jointly across 13 Indic scripts within a single framework. UniLipi directly handles realistic manuscript conditions, including extreme variation in line geometry, large variation in line length, and partial interruptions caused by non-textual manuscript entities such as holes, stains, or pictorial illustrations. To operate effectively under ultra low-resource conditions, the model leverages script-aware synthetic manuscript data generation, substantially reducing reliance on large volumes of real annotated data. Beyond historical manuscripts, we show that UniLipi serves as an effective foundational pretrained model. Specifically, its learned representations enable good OCR performance for contemporary Indic handwriting and extend to several non-Indic scripts, including Tibetan, Italian, Latin, and Chinese scripts. In addition to transcription, UniLipi predicts script identity and per-line native character counts, supporting practical manuscript cataloging workflows.

0.35

Statistical Models for Automatic Fingering-Annotated Piano Sheet Music Transcription

cs.SD ·2026-09-23 ·Daniel Penner, Abram Hindle
similarity 0.59
Abstract · 9 pages, 5 tables, 4 figures

Machine learning tools have significantly aided automatic piano music transcription; however, this domain has focused primarily on accurately predicting the pitches and timings of played notes. To produce sheet music for the piano, notes must be separated into two staves, one for each hand, and good sheet music often contains fingering annotations to guide the player when sight-reading or learning fast or complex pieces. We propose 8 statistical approaches for combined hand and fingering annotation of transcribed piano notes, including baseline hidden Markov models, rule-based methods, a synthesis of existing approaches, and N-gram language models. Furthermore, we develop a pipeline for complete transcription from piano audio to fingering-annotated sheet music. Evaluations with the PIG dataset demonstrate that our Synthesis model achieves a hand separation accuracy of 90.8\% and a joint hand and finger annotation accuracy of 56.6\%. These approaches serve as a new baseline for further research into this problem, while our pipeline demonstrates the feasibility of a combined system for automated note transcription, hand separation, and fingering annotation.

0.35

OmniScore-Bench: Evaluating Vision-Language Models on Musical Score Reading

cs.AI ·2026-11-13 ·Congren Dai, Zhancheng Guo, Danni Zhao, Wenchu Wang, Xiaobing Li +1 more
similarity 0.59
Abstract

Vision-language models (VLMs) can answer questions about musical scores, but their ability to recognise notation, recover its structure and interpret its musical meaning requires separate evaluation. Existing optical music recognition (OMR) resources establish detection and transcription measurements; a transcription score alone leaves the strengths and bottlenecks of a VLM score reader unresolved. We introduce OmniScore-Bench, an evaluation of OMR capabilities in VLMs through 3,000 questions spanning six aspects and 24 task subtypes. Each question specifies a visible scope, reference annotations and an answer schema, enabling automatic evaluation from symbol localisation to musical consistency and evidence-linked question answering. Our assessment of twelve VLMs reveals operation-specific capability profiles and different transcription leaders across excerpt, single-page and multipage tasks. Controlled analyses examine the roles of score images, auxiliary givens and additional page context. On the tested subset, image removal exposes visual dependence in pitch and repeat interpretation while measuring how much performance remains with public context alone. Separate format and content analyses distinguish answer-format violations from musical errors. OmniScore-Bench supports diagnosing which score-reading operations VLMs perform reliably and which require improvement.

0.13

TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data

cs.SD cs.AI ·2026-09-01 ·Jianhuai Hu, Yashan Wang, Shangda Wu, Zhancheng Guo, Shijie Liang +5 more
code/data similarity 0.59
Abstract

Generalizable Audio-to-Score (A2S) transcription is fundamentally constrained by the severe scarcity of high-quality, real-world paired data. Relying solely on existing human-annotated datasets often restricts the generalization of A2S models, limiting their efficacy primarily to single-instrumentation domains. To break this dependency on scarce real-world data, we introduce TUTTI (Transformer for Unified audio-To-score Transcription trained on Synthetic multi-Instrumentation Data), a pre-training paradigm driven by a purely synthetic, large-scale dataset. Rather than using human-composed scores, we leverage a symbolic music generation model to generate a massive, highly scalable multi-instrumentation corpus and create audio-score pairs with expressive acoustic characteristics. Capitalizing on the generated data, we employ a standard Transformer encoder-decoder architecture. We empirically demonstrate that pre-training a unified attention-based model on generated, multi-instrumentation data yields a consistently stronger foundational representation than single-instrumentation training. When fine-tuned with downstream real-world datasets, TUTTI outperforms previous approaches, establishing new overall state-of-the-art results across various A2S baselines. Notably, TUTTI shows remarkable cross-instrument transferability, effectively adapting to unseen instruments with highly competitive performance. The source code and the TuttiCorpus dataset will be made publicly available at https://github.com/a-musiclover/TUTTI.

0.54

Measuring Annotation Efficiency for Handwritten Devanagari Recognition: Sample-Complexity Curves for Four Pretraining Regimes

cs.CV cs.LG ·2026-09-15 ·Manglesh Kumar Pandey, Sumit Kumar Banshal
similarity 0.58
Abstract

To train handwritten text recognition systems we need word images and their corresponding transcriptions, and these transcriptions are produced manually. For a script that can be read by only a small number of specialists, this manual transcription is a limitation, because the trained models are supposed to save the time of those same specialists. A relevant question therefore arises: how many transcriptions are needed before a recogniser becomes useful, and how much of that cost can pretraining remove? In this study the answer is measured directly for handwritten Devanagari. We keep the recogniser, optimiser and evaluation protocol the same and change only the number of real transcribed words used for fine-tuning across nine budgets from 10 to 4,000 and four initialisation regimes, with six seeds at every point. The resulting curves are then converted into annotation-equivalent terms. A CER of 0.50 is reached by supervised synthetic pretraining using only 81 transcribed words, whereas random initialisation requires 355, which gives a label multiplier of 4.40 [3.56, 4.99]. There is a zero-shot reference point as well: with no real transcribed words at all, this pretraining is worth about 136 of them. This advantage gets smaller as the target accuracy improves, and at the most demanding target we measure, it cannot be distinguished from no saving at all. A fourth arm in which only the encoder is transferred separates the effect of the pretraining method from that of transfer scope, and masked image modelling is observed to transfer negatively over a bounded range of budgets. We emphasise that the scarcity in this study is constructed by subsampling a large corpus.

0.18