CS Paper Radar

Data-Driven Glyph Positioning for Handwritten Music Synthesis

Elona Shatri, Jiří Mayer, Jan Hajič

cs.CV ·submitted 2026-10-20 20:25 UTC · v1
published

Abstract

Synthesised data is a common strategy to address the lack of annotated handwritten music scores in Optical Music Recognition (OMR). Smashcima is a recent synthesis framework that assembles pages of handwritten scores by sampling glyph images from MUSCIMA++ and positioning them with hard-coded rules. In this work we ask whether these rules are grounded empirically and whether a learned model for context-dependent positioning can further improve synthesis quality. We extract inter-glyph distributions from around 285 thousand annotated relationships across MUSCIMA++ and OmniOMR, finding that several key parameters deviate systematically from real handwriting. Replacing these constants with empirical means reduces positioning RMSE by up to 60%. We then train a Graph Neural Network (GNN) on the Music Notation Graph structure to predict context-dependent offsets, achieving an additional 50 to 85% RMSE reduction on stems and beams by modelling how neighbouring glyphs constrain placement. Cross-dataset analysis shows that spacing differences between the two corpora are explained by note density (3.3), not style, and that duration-proportional spacing does not hold in handwriting (R2 = 0.001). In downstream evaluation, a YOLOv8 detector pretrained on GNN-positioned synthetic data achieves +30% mAP50 over the baseline and +11% over empirical means when fine-tuned on 200 real handwritten staves, showing that positioning accuracy matters for downstream OMR.

Comment
10 pages, 1 biblio page, presented at The 20th International Conference on Document Analysis and Recognition

Closest papers

Nearest neighbours by meaning across the whole index.

Full-Page Optical Music Recognition of Handwritten Monophonic Scores

cs.CV cs.AI ·2026-09-04 ·Adrian Rosello, Antonio Ríos-Vila, David Rizo, Jorge Calvo-Zaragoza
similarity 0.66
Abstract

Full-page end-to-end Optical Music Recognition seeks to transcribe entire music pages directly into symbolic notation, avoiding the limitations of traditional pipelines that rely on accurate staff segmentation. Recent Transformer-based architectures have achieved strong performance on typeset scores, relying on large-scale synthetic data for pretraining. However, their applicability to handwritten music remains largely unexplored. In this work, we study full-page transcription on handwritten monophonic collections and analyze the impact of synthetic pretraining in this setting. To investigate which factors are most relevant during pretraining, we introduce a generator capable of producing visually coherent full-page scores in both typeset and handwritten styles. Experiments on three real handwritten datasets provide a comparative evaluation of several full-page pipelines and different synthetic pretraining strategies. The results suggest that the benefits of synthetic pretraining are primarily associated with learning structural layout conventions rather than with visual similarity to the target handwriting.

0.21

Statistical Models for Automatic Fingering-Annotated Piano Sheet Music Transcription

cs.SD ·2026-09-23 ·Daniel Penner, Abram Hindle
similarity 0.65
Abstract · 9 pages, 5 tables, 4 figures

Machine learning tools have significantly aided automatic piano music transcription; however, this domain has focused primarily on accurately predicting the pitches and timings of played notes. To produce sheet music for the piano, notes must be separated into two staves, one for each hand, and good sheet music often contains fingering annotations to guide the player when sight-reading or learning fast or complex pieces. We propose 8 statistical approaches for combined hand and fingering annotation of transcribed piano notes, including baseline hidden Markov models, rule-based methods, a synthesis of existing approaches, and N-gram language models. Furthermore, we develop a pipeline for complete transcription from piano audio to fingering-annotated sheet music. Evaluations with the PIG dataset demonstrate that our Synthesis model achieves a hand separation accuracy of 90.8\% and a joint hand and finger annotation accuracy of 56.6\%. These approaches serve as a new baseline for further research into this problem, while our pipeline demonstrates the feasibility of a combined system for automated note transcription, hand separation, and fingering annotation.

0.35

MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control

cs.SD eess.AS ·2026-10-19 ·Lynn Yi, Zihan Xiong, Junjie Cao, Xiaolong Weng, Yingzhe Ma +15 more
code/data similarity 0.62
Abstract

A cover song re-renders an existing piece, most commonly by preserving its tonal content -- melody and chord progression -- while reshaping other musical attributes. This suggests a natural formulation for generative cover synthesis with pretrained music language models: explicitly control tonal content, while specifying lyrics and style through text. Existing approaches condition on frame-level features, tightly coupling generation to the reference's timing and structure, and limiting flexibility such as tempo variation, structural rearrangement, or partial conditioning. To this end, we present MuLaCover, a cover generation model that conditions a pretrained text-to-song foundation model on an editable symbolic lead sheet. Our design enables alignment-free control: generated outputs preserve the tonal signature of the source without frame-level alignment, allowing flexible timing, structure, and segment-level generation. We further introduce a training curriculum with partially mismatched audio-symbolic pairs, improving diversity and robustness. We evaluate MuLaCover with objective metrics, standard subjective ratings, and in-depth expert interviews that provide fine-grained diagnostic insights. Experiments show that MuLaCover achieves state-of-the-art performance among open-source systems and is competitive with leading commercial models such as Suno-v5.5. Audio samples and code are available at our project page: https://mulalabs.ai/mulacover/

0.23

Getting Motif-ated: Controllable AI Compositions from Injected Motif Prompts

cs.LG cs.AIcs.MMcs.SD+1 ·2026-09-26 ·Chao Peter Yang, Cynthia Rudin, Yue Jiang, Simon Mak, Stephen Ni-Hahn
published similarity 0.60
Abstract · NeurIPS 2026, Creative AI Track

Deep learning has transformed symbolic music generation by borrowing the training paradigms of large language models, with systems such as NotaGen now producing complete, stylistically convincing classical scores from a short prompt. These systems could become powerful creative partners, helping musicians generate endless possibilities. However, current systems expose almost no control handles on the music itself. In principle, control handles could be built into a foundation model trained from scratch, but this is rarely practical without massive amounts of quality annotated data and compute resources. We therefore present MotiGen, a recipe for retrofitting pretrained symbolic music models to use new instruction prompts. MotiGen injects a musical motif as a structured prompt line, reinforces it with a scalar attention bias toward the motif tokens, and learns the association with a two-phase curriculum. First, it learns from focused excerpts cropped around motif occurrences in the training data, then full scores including the motifs. Our experiments show that our model composes with the prompted motif in over 92.3\% of generated pieces. Generated pieces using a variety of motifs are included in our sample site: https://motigen-site.github.io/.

0.23

Japanese Handwriting Generation Based on Human Preference Feedback Using Direct Preference Optimization

cs.CV ·2026-10-10 ·Wakana Kuwata, Ryota Mibayashi, Masanori Tani, Hiroaki Ohshima
published similarity 0.59
Abstract · Accepted for publication at BESC 2026

We propose a Japanese handwriting generation framework that integrates human preference feedback using Direct Preference Optimization (DPO) to align generative models with human cognitive and stylistic preferences. While font generation technologies leveraging Generative Adversarial Networks and diffusion models have advanced significantly, applying these approaches to handwriting generation presents unique challenges. Font generation is a deterministic process yielding consistent styles. Handwriting generation, on the other hand, is inherently non-deterministic. In handwriting, glyph shapes never match exactly even when the same writer writes the same character. We define this shape difference as handwriting variability. Although this variability is essential for conveying the natural quality of human writing, it complicates model training and substantially increases data collection costs to capture diverse distribution patterns. To address these limitations, we incorporate human perceptual feedback into the training loop of handwriting generation. While collecting extensive handwritten data is costly, evaluating pairwise preferences between two generated samples remains very easy for humans. Applying DPO, an efficient RL-free preference optimization method, to our generation model enables high-quality handwriting generation without relying on large-scale dataset collection. Experimental results across four distinct Japanese handwriting styles demonstrate that DPO-based training enhances overall structural accuracy. However, in highly complex styles with significant variability, such as brush pen writing, the model fails to capture style-specific features. These findings highlight a critical challenge in extending DPO to complex handwriting styles.

0.30

Reference-Free Singing Pitch Correction via Music-Constrained Sequence Editing

cs.SD eess.AS ·2026-10-08 ·Biao Dong, Jiajun Li, Binzhen Zhu, Mingwei Yi, Tong Liu +3 more
similarity 0.58
Abstract · Submitted to ICASSP 2027

Existing singing pitch correction approaches rely on target melodies or accompaniment tracks, which may be unavailable in practice. We formulate reference-free singing pitch correction as a music-constrained sequence editing task that determines whether and how each note should be corrected from the input performance alone. A pretrained symbolic music encoder with lightweight singing-domain adapters produces contextual representations of the singing MIDI. Based on these representations, two lightweight correction heads jointly model correction necessity and signed pitch modification through a factorized pitch-editing distribution. An input-dependent tonal prior derived from the estimated key distribution then reranks the candidate offsets, favoring tonally compatible corrections without target melodies or ground-truth key annotations. Experiments on real paired amateur and professional singing recordings show that the method improves note-level pitch accuracy from 73.25 to 83.43, outperforming a context-based baseline by 3.99 percentage points while balancing error correction and preservation of correctly performed notes.

0.20

Assessing Alignment in Automatic Piano Transcription Datasets: A Benchmark for Scoring Methods

cs.SD eess.AS ·2026-11-10 ·Lucas de Souza Brandão, Lucas Gris, Joaquim Cavalcante, Chukwuemeka Nkama, Andrea Poltronieri
benchmark similarity 0.57
Abstract · 6 pages

Automatic Music Transcription (AMT) relies on paired audio--MIDI datasets, implicitly assuming near-perfect temporal alignment. However, widely used piano datasets have been shown to contain alignment errors that may bias model evaluation and training. We propose a framework for evaluating audio--MIDI alignment scoring methods through controlled temporal, pitch, and structural MIDI corruptions. We also introduce STI, a scoring method built on AMT models, and benchmark it against existing alignment scoring approaches. Our results show that STI, together with methods based on CENS and spectrograms, achieves the strongest overall performance, with STI exhibiting the highest sensitivity to fine-grained temporal misalignments, the main alignment error in AMT datasets. We then apply the best-performing methods to widely used piano AMT datasets and identify trends in their relative alignment quality. These results support more informed dataset curation, model training, and AMT evaluation.

0.12

DuET: Note-Frame Dual Encoding for Piano Performance-to-Score Transcription

cs.SD cs.LGeess.AS ·2026-10-15 ·Wei Zeng, Ye Wang
code/data similarity 0.57
Abstract · 14 pages, 3 figures

Transcribing expressive piano performances into scores requires interpreting individual note timings within an underlying metrical structure. However, score attributes are associated with notes, whereas beats may occur without coincident note onsets. We introduce DuET, a Dual-Encoder Transformer that jointly learns note-level performance MIDI-to-score (PM2S) transcription and frame-level beat and downbeat tracking. Bidirectional cross-attention connects note and frame encoders, while an autoregressive decoder alternates between their representations to predict structured score attributes. We further introduce TimeRoPE and Time-Pitch Rotary Positional Encoding (TPRoPE) to establish a shared performance-time reference and explicitly encode relative temporal and pitch relationships. Evaluations on ASAP, a classical piano dataset, and our curated Gangqinpu dataset, which spans multiple genres with a focus on popular piano music, demonstrate improvements in score timing and notated duration. Compared with the strongest baseline on each dataset, DuET reduces MUSTER mean error from 6.56% to 3.64% on Gangqinpu and from 11.30% to 11.22% on ASAP. Ablations support the benefits of frame-level context, time-based positional encoding, and decoder-history masking. Code, weights, and demos are available at https://github.com/wei-zeng98/DuET.

0.29