CS Paper Radar

TimeCues Studio: A Workspace for Music Annotation and Algorithm Prototyping

Sapir Caduri, Yoav Goldberg

cs.SD cs.HCcs.LGcs.MM ·submitted 2026-09-09 15:35 UTC · v1
top venuecode/data

Abstract

Multimedia applications require precise music annotation-labeled positions, segments, or loops-placed by hand or algorithmically. Machine-learning algorithms are scalable and effective but need annotated training data, scarce for many tasks. TimeCues Studio is an open-source workspace where algorithm-development teams annotate a music corpus, compare detection algorithms against those annotations, and prototype new ones. Unlike existing tools built for a single track at a time, TimeCues targets teams annotating whole collections, tightly integrated with algorithm development. Annotators place several marker types-each supporting ambiguity-aware labeling-on a grid-locked timeline that visualizes many music features, including separated audio stems. The same timeline drives an algorithm-comparison engine with bundled baselines, a Python sandbox for prototyping new models, and an ambiguity-aware evaluator that honors the structured fields. The same visualization suits solo annotators on music-sync projects. TimeCues is MIT-licensed and deploys via one Docker Compose command.

Comment
8 pages, 2 figures, to appear in Proceedings of the 34th ACM International Conference on Multimedia (MM '26)
DOI
10.1145/3767308.3834750

Closest papers

Nearest neighbours by meaning across the whole index.

LayerIt: Towards a Framework for Time-Aligned, Composable Music Visualizations

cs.SD cs.MM ·2026-10-02 ·Fernando Azeredo, António Sá Pinto
publishedcode/data similarity 0.67
Abstract · Accepted as a Late Breaking Demo (LBD) at the International Society of Music Information Retrieval conference (ISMIR) 2026

Music information retrieval often relates signal-derived, algorithmic, and symbolic information across different coordinate systems. Existing visualizations typically leave notation separate from physical time, while composites that combine them are assembled by hand. We present LayerIt, an open-source Python library for composing independent representations with notation on a shared performance-time axis. Given a score warped into performance time and a note-level alignment, LayerIt emits a single SVG that preserves the score's MEI structure and keeps added components identifiable and restylable. We demonstrate this through a challenging beat-tracking analysis, in which tracker output, signal representations, and notation can be inspected together to locate metrical disagreement and other errors against both notated structure and performed time.

0.37

Building a Dataset for Music Sample Identification

cs.SD ·2026-09-14 ·R. Oguz Araz, Xavier Lizarraga, Xavier Serra, Dmitry Bogdanov
code/datahot direction similarity 0.63
Abstract · Extended Abstracts for the Late-Breaking Demo Session of the 27th Int. Society for Music Information Retrieval Conf., 2026

Sample identification (SI) is the task of matching an element of a musical work to its musically transformed versions used to create new works. The task has received little attention and lacks large-scale publicly available data. In this work, we mine sampling annotations from a music database and split them for training and evaluation. The resulting dataset is nearly three orders of magnitude larger than the existing SI benchmarks, with training, validation, and test sets of 114 k, 6 k, and 10 k tracks. We find that naively splitting the annotations places the same tracks in different sets. To avoid this, we construct a graph from the annotations and split it over connected components. We further find that a single mega-component contains half of the annotations, making component-wise splitting incompatible with balanced splits; we trim it, yielding a leakage-aware pipeline. We share the dataset for non-commercial scientific research purposes only and make the data-analysis and splitting code publicly available. We hope that our work fosters research on SI.

0.61

FaiSonic MMD: A popular music dataset with temporally aligned multidimensional annotations for multitask music analysis

cs.SD ·2026-11-10 ·Tieyao Zhang, Yuke Liu, Jiaxing Yu, Xinda Wu, Genfang Chen
datasetcode/data similarity 0.61
Abstract · 8 pages, 3 figures, 2 tables. Dataset and code available at https://github.com/Kreuzter0421/FaiSonic-MMD

Temporal analysis in music information retrieval concerns the positions of musical events and structures within recordings. Joint modeling has demonstrated the value of combining musical descriptions, but existing resources differ in joint coverage and expert verification across dimensions, constraining detailed joint supervision and the analysis of relationships on the same recordings. To support multitask modeling and the study of relationships among musical dimensions in popular music, we introduce FaiSonic Multidimensional Music Dataset (FaiSonic MMD), containing 500 songs totaling approximately 32.86 hours, comprising 250 Chinese-language and 250 English-language songs. Professional musicians annotate phrases, sections, chords, and rhythm in real recordings, followed by independent review. Events and intervals across dimensions share a recording-based time reference while retaining their respective musical granularity and boundaries, allowing direct comparison across musical descriptions. We describe the annotations, collection procedure, and statistics, and establish a comprehensive multitask baseline for phrase boundaries, section boundaries, section functions, chords, beats, and downbeats. We report reference performance for all prediction targets. FaiSonic MMD provides shared expert annotations and accompanying benchmarks for temporal music analysis.

0.23

ProCraft: Instruction-Driven Music Production from Free-Form Text

cs.SD eess.AS ·2026-11-05 ·Yuan-Chiao Cheng, Yi-Hsuan Yang, Vincent K. M. Cheung
similarity 0.61
Abstract · 27 pages, 6 figures, 11 tables

Music production typically involves multiple editing operations that each require specialized tools and expertise. Existing text-guided music editing systems support few operations or instruments, operate on short clips, or handle only one operation per instruction, unlike real workflows where several changes are requested at once. Here, we present ProCraft, a lightweight 0.5B-parameter autoregressive codec language model that edits 30s music excerpts end-to-end from free-form instructions, including compound requests. ProCraft supports seven operations across eight instruments, from tasks with a unique target (e.g., track removal) to creative ones (e.g., accompaniment generation). We align intermediate representations with pretrained music embeddings and pitch and instrument targets to preserve content outside the edit, and gated cross-attention in every decoder layer to keep generation tied to the instruction. To train on compound requests, we introduce On-the-fly Editing-pair Synthesis, which generates paired instructions and audio combining track-level operations with effect changes. ProCraft performs competitively against task-specific systems on single-task editing, and outperforms four free-text editors on compound instructions for track removal, extraction and denoising on real (MoisesDB) and synthetic (Slakh) recordings. In a user study (16 listeners), ProCraft was rated highest on removal and denoising and on-par with a 16-times larger model on extraction and instrument transfer. Ablations further show that representation alignment and cross-attention reduce leakage from unwanted tracks and raise editing success rates, especially on out-of-domain data.

0.11

One Timeline, Many Renderings: A Wolfram Language Paclet for heterogeneous musical output

cs.CL cs.SD ·2026-08-25 ·Francesco Vitucci, Michele Lorusso, Francesco Scagliola
publishedcode/data similarity 0.60
Abstract · Accepted at the International Csound Conference (ICSC) 2026

One algorithmic composition may require a Csound score, engraved notation, real-time control, and a rehearsal click. Authored separately, their timelines drift. Temporal System is a Wolfram Language paclet that instead compiles one immutable store of typed entities on a rational beat timeline through backend-specific contracts. It emits Csound synthesis, beta MusicXML 4.0, OSC control, and click artifacts that remain synchronized because they share that store. Conversion to seconds, samples, or hertz occurs only at render time. Csound notes use stable named instruments in external .orc files; curves become k-rate signals declared against score p-fields. The click backend derives rehearsal audio from the same meter and tempo and reuses the Csound serializer. We describe the temporal, semantic, and rendering-contract layers, their practical trade-offs, and the limits of this proprietary authoring environment within an otherwise open-source ecosystem. The archived supplement exposes the reported outputs pending paclet release.

0.42

Gangqinpu: A Multi-Genre, Multi-Task Piano Dataset with Aligned Audio, MIDI, and Musical Scores

cs.SD cs.AIeess.AS ·2026-10-26 ·Wei Zeng, Anvay Mathur, Ye Wang
datasetcode/data similarity 0.59
Abstract · 9 pages, 3 figures

Aligned audio, performance MIDI, and musical scores provide valuable supervision for music analysis and performance modeling, yet datasets that jointly align all three modalities remain scarce and limited in scale and genre diversity. We introduce Gangqinpu, a multi-genre piano dataset containing 18,786 score-performance pairs and 1,007.1 hours of source audio, with automatic note-level alignments and derived beat/downbeat annotations. It spans diverse genres, including pop, classical, jazz, folk/traditional, rock, and blues. Our automated pipeline transcribes piano audio, aligns the resulting MIDI with musical scores, assigns genre labels, and filters out non-expressive performances with a score-informed binary classifier. We further construct joint splits with ASAP using conservative score-similarity grouping to reduce data leakage. We evaluate Gangqinpu on performance MIDI-to-score transcription, expressive performance rendering, and beat tracking. Downstream experiments demonstrate the value of Gangqinpu as complementary training data, with cross-dataset gains in beat tracking and primarily in-domain benefits for transcription and expressive performance rendering. The code and dataset are publicly available at https://github.com/wei-zeng98/gangqinpu.

0.22

MUB: An Identity-Masked Benchmark for Music Understanding in Multimodal Language Models

cs.SD ·2026-10-16 ·Kun Liu, Lyra Tao
benchmark similarity 0.59
Abstract

Evaluating musical perception in multimodal language models requires separating audio evidence from answers recoverable through metadata or external tools. We present the Music Understanding Benchmark (MUB), a locally assembled collection of 500 identity-masked recordings with existing annotations for tempo, key, time signature, and structural intervals. The collection covers all 24 major/minor key combinations, tempos of 47--180 beats per minute, and 4,676 annotated intervals. Original titles, artists, and platform identifiers are excluded from evaluation inputs. Relative to recent music question-answering and temporal-grounding work, MUB emphasizes scalar tempo prediction and full-recording interval outputs, together with a documented metadata-masking protocol. Saved baseline predictions yield 53.6\% tempo accuracy for a simple DSP estimator and 98.4\% time-signature accuracy for a constant 4/4 predictor. We identify important limitations, including an asymmetric legacy key metric, unverified independent annotation agreement, extreme metre imbalance, and unresolved recording-level rights clearance. No contemporary multimodal language model comparison has yet been completed. MUB is therefore an evaluation resource and protocol under validation, rather than a completed demonstration of model capabilities.

0.09

Unified Music Identification for Tracks and Versions

cs.SD ·2026-08-20 ·R. Oguz Araz, Joan Serrà, Yuki Mitsufuji, Xavier Serra, Dmitry Bogdanov
publishedbenchmark similarity 0.59
Abstract · Accepted to ISMIR2026

Given a music database, track identification (TI) retrieves the exact track matching an audio excerpt, whereas version identification (VI) retrieves its musical versions. Traditionally, the two tasks have been addressed separately. However, as every track is its own closest version, we investigate whether VI can subsume TI. This requires VI systems to be robust to both signal manipulation and audio degradation. We therefore propose a unified benchmark that evaluates accuracy and robustness on each task. Comparing seven existing models on this benchmark, we show that none of them are both accurate and robust on both tasks. We then train a baseline model targeting both tasks and show that a unified system is possible with 10 s TI queries. Lastly, we characterize the two retrieval constraints that limit our model's TI performance. We envision extending this unification to other music identification tasks.

0.41