CS Paper Radar

Structural and Representational Agency in LLM-Driven Timbre Generation and Interpolation

Joseph Cameron, Alan Blackwell

cs.SD cs.HCcs.LGcs.MMeess.AS ·submitted 2026-10-08 12:37 UTC · v2
top venueunusual

Abstract

Generative creative interfaces routinely report the user's position within a space of possibilities. Such displays imply that the quantities shown were measured from the artefact; in a growing class of systems they are instead predicted by a language model describing its own output, and the two are indistinguishable on screen. Most creative domains lack an agreed measurement against which such a self-report can be checked; audio admits one. In a measurement study of a working LLM-driven timbre interface, what a user can cause proves separable from what the interface reports. The first is exactly characterisable: the reachable set is a zonotope in parameter space whose dimension increases by one per sound placed, so the space the model may act within is demonstrably the user's own. The second is inaccurate. The displayed position differs from measurement by approximately three times the width of that region, and the error has two independent sources: model-predicted coordinates, whose accuracy varies twofold between the two models evaluated, and the interpolation step, which arises from combining synthesiser parameters and is unaffected by model quality. End to end, 27 of 36 refinement instructions produced no perceptible change, and under the weaker model the facility performed no action while reporting convergence. Effective agency (what a user can cause) and epistemic agency (whether they can know it) are therefore separable and independently engineered: structurally enforced constraints survive model failure, whereas guarantees resting on the model's own report do not. The failure mode that remains is silence presented as success. We close with a short protocol for building and auditing such displays.

Comment
11 pages, 2 figures, 4 tables. Accepted at NeurIPS 2026 (Main Conference, Creative AI Track)

Closest papers

Nearest neighbours by meaning across the whole index.

Do Audio Language Models Hear and Read Distinctive Features Alike?

cs.CL cs.LGcs.SDstat.AP ·2026-09-24 ·Yuanhao Chen, Peter Chin
similarity 0.57
Abstract

Audio language models pass speech and text through a single decoder. We ask whether that decoder represents a distinctive feature in the same direction when a phoneme is heard and when it is read. For minimal pairs of phonemes differing in one feature, we take the offset between the two members' mean representations. Averaging those offsets gives a direction for each stream, and we measure the cosine between the two. Because the two streams already agree about arbitrary phoneme pairs, we compare every measure against a reference built from random pairings rather than against zero. We apply this to 6 models, 7 features and 15 languages from 11 families. Only voicing in the two Qwen2.5-Omni models exceeds that reference after correction for multiple testing, and the reference varies by a factor of seven between models. In three of the six models, voicing has one direction in audio across the 14 languages with enough minimal pairs to measure it, and every language pair agrees in two of them. The model family, not the model size, predicts which stream represents a feature.

0.33

Derivation Depth Exposes Reasoning Failures in Tool-Augmented Music Analysis

cs.SD ·2026-10-21 ·Rupali Rajesh, Sethuraman T, Savya Khosla, Srinidhi Sunkara, Aditya Shanmugham +2 more
similarity 0.56
Abstract

Audio-language models remain unreliable at extracting basic musical properties from audio, motivating pipelines that place a music information retrieval (MIR) tool between the audio and the model. The tool reports measurements such as keys, chords, and structural similarity, and the model is expected to derive the answer. We test whether it can. Holding the musical information fixed, we vary derivation depth: the number of reasoning steps between the tool output and the answer. When the report states the conclusion, all five models score near 100%, and they follow it even when it is wrong. One step away, accuracy drops. On modulation, open models fall to 57-81%, although they answer that single step correctly when asked directly, and reach 0% at greater depths. Frontier models degrade less, but still degrade. Adding the audio does not recover the loss. The bottleneck in tool-augmented music analysis is not access to musical information but reasoning from it, so tool design must account for how much derivation it leaves to the model.

0.13

MI-MIDI 2: From Observation to Control with Sparse Autoencoders in Text-to-MIDI Generation

cs.SD cs.AI ·2026-10-23 ·Jakub Poćwiardowski, Mateusz Modrzejewski
similarity 0.56
Abstract · 20 pages, 6 figures, 24 tables. Atlas with audio examples: https://jpocwiar.github.io/MI-MIDIv2-Demo/

Sparse autoencoders (SAEs) can expose features of a generative model without predefined labels, but in language models the evidence used to interpret these features has proven an unreliable guide to which of them steer the output. Whether interpretations of SAE features predict control in music generators has not, to our knowledge, been tested. We develop an end-to-end protocol for discovering and evaluating SAE features in text-to-MIDI generation and apply it to text2midi and MIDI-LLM. The protocol characterizes features through correlations with musical metrics, shared instrument programs, and automated descriptions, each with its own control: note-density adjustment, matched inactive excerpts, and permutation calibration. It then tests candidates by intervention, measured directly in the generated MIDI against unmodified generation and a matched random direction, ranks them in screens, and confirms selected features on new prompts. Evidence that survives these controls points to features that steer the output. Descriptions exceed permutation controls for instrumentation, and a feature whose description names a synth-lead program introduces that program in a prespecified confirmation and in a replication on disjoint prompts and reappears in independently trained dictionaries. Features selected by density-adjusted correlations show a small positive shift of their target metric as a family, whereas features selected by raw correlations show none. An atlas provides MIDI, piano-roll, and audio examples for every feature of the targeted screen.

0.19

Design Creativity Bench: Measuring creativity in LLM-Generated UI

cs.HC cs.AI ·2026-10-08 ·Aman Rusia, Abhijit Bhole, Prashank Gupta, Dipanjan Dey
benchmark similarity 0.54
Abstract · v2: added link to live leaderboard (https://kombai.com/benchmark/); clarified that appropriateness results apply to the evaluated models. Re…

As leading LLMs improve on capability evaluations, their limitations in producing creative outputs on design tasks remain insufficiently characterised. Our work introduces Design Creativity Bench, a benchmark that evaluates diversity and appropriateness in UI designs. It measures distinctiveness among models on the same prompt (originality), how much a model's designs change between two prompts for the same UI goal in different product domains (creative range), and the share of a brief's acceptance criteria each design meets (appropriateness). Originality is 0.592 for same-prompt design pairs from different models (95% CI [0.582, 0.602]), far below the 0.764 for same-prompt human-model pairs (95% CI [0.751, 0.778]). Creative range is 0.581 across models (95% CI [0.567, 0.597]), against 0.902 for human designs (95% CI [0.884, 0.919]). Appropriateness is above 90% for every model in our test set, and the best model reaches 99.2%, slightly above the 98.0% for human designs. Our work shows that the default output of LLMs, though generally appropriate, is substantially more repetitive than the human baseline. This calls for strong measures to address the issue. A live leaderboard with newer models is maintained at https://kombai.com/benchmark/.

0.35

Is Imagination Derived from Hallucination? A Cross-Taxonomy Evaluation of Imagination and Hallucination in Large Language Models

cs.CL ·2026-08-26 ·Zixuan Tang, Hongzong Li, Shuxin Zhuang, Dapeng Wu, Zi Liang
benchmark similarity 0.53
Abstract

Imagination performs as a high-level function of large language models (LLMs) which determines the potential of how an LLM creates unseen or creative content. While existing works have built a rich family of creativity benchmarks for this ability, they only measure how far an output departs from common answers and never check whether the departure is licensed by the prompt. Moreover, hallucination, the closest neighbor of imagination, is always measured in a separate pipeline on different generations, so the influential claim that imagination and hallucination stem from the same generative mechanism has never been directly testable. In this paper, we propose Whiteboard, the first LLM imagination evaluation benchmark. Its design follows the authoritative cognitive instruments developed to measure human imagination: seven mechanism-grounded imagination subtypes are adapted from classic paradigms, then crossed with ten support-boundary hallucination subtypes and scored jointly on the same generation. Different from previous creativity or hallucination benchmarks, Whiteboard gates every imagination score with an explicit support check and computes both axes deterministically through an auditable atom matrix, with no LLM judge on the primary path. The full Whiteboard item bank contains 1,660 prompts; on its shared 80-item anchor set, we evaluate 79 state-of-the-art LLMs and validate the instrument against 13,280 human judgments. Additionally, we further explore whether imagination derives from the same generative tendency as hallucination and what key factors shape it. Our analysis indicates a counterintuitive correlation between hallucination and imagination: Most of the subtype couplings are negative, every one of the anchor items reproduces the negative coupling on its own.

0.20

Automated Parameter Sensitivity Ranking in Audio Synthesis

cs.SD ·2026-10-28 ·Ogulcan Kibar, Baris Ozcan
similarity 0.53
Abstract · Submitted to the Journal of the Audio Engineering Society (JAES). 8 pages, 4 figures, 6 tables

Software synthesizers expose dozens of coupled controls, many of them redundant or perceptually inert at typical operating points. This paper presents a two-stage, measure-and-validate framework that ranks parameters by their influence on the audio output and then checks that ranking against an independent, variance-based experiment. Every rendered clip is normalized to -23 LUFS and described by five spectral fingerprints, so that gain shifts cannot masquerade as timbral change. The Sweep Test scores each parameter by spectral deviation over a 101-step sweep; the Sobol Test scores it by summed per-cell variance over a 128-point Sobol excitation. Across five architecturally distinct synthesizers (TorchSynth, Pyo, Pedalboard, Csound, and FluidSynth) spanning 48 to 76 parameters, the two tests agree at Spearman ρ= 0.82-0.96 (p-values \le 7.2 \times 10^{-15}) on the four engines that render deterministically, and at ρ= 0.32 on the one with process-level render noise--a diagnostic of that engine rather than of the method. The strong correlations survive reducing the Sobol budget from 128 to 32 points, and the complete analysis costs a one-time offline budget of about 66,000 renders.

0.35

A Calibrated Instrument for Measuring How Inference Optimizations Affect Output Quality

cs.CL cs.LG ·2026-09-16 ·Jerry Kaplan
benchmarkcode/data similarity 0.53
Abstract · 23 Pages. 6 tables in main text,5 tables in appendices. Code, prompts, and result files at https://github.com/jerrykaplan/Calibrated-Instrum…

Large language model optimization is an active research area, spanning quantization of model weights, early-exit methods for skipping layers, and speculative decoding. Each track uses its own quality measures, typically an idiosyncratic benchmark score. Few approach the measurement precision required by other scientific disciplines. We propose a rigorous methodology for measuring output quality, suitable for cross-system and cross-technique comparison. We score outputs with an LLM as a judge, but calibrate the judge formally: we compare its scores on two ordinary runs of a model given the same prompts, verifying that it shows no systematic preference between statistically equivalent outputs and measuring its per-sample noise. Each design also includes a 'null' condition, provably identical in distribution to the unmodified model, whose measured difference must be zero. With this one instrument we measure several acceleration techniques on the same prompts, so their quality costs can be compared. Perceived quality proves highly dependent on the domain of discourse. A 4-bit model was indistinguishable from its 16-bit original down to our design's +/-0.3-point resolution, in English prose and Chinese alike. At 3-bit precision the same prompts lost 0.5 points in English prose, 0.9 in Chinese, and 1.1 on multi-step math; early exit that cost 0.7 points on prose cost 2.5 on math, cutting correctly solved problems from 19 of 27 to 6. The pattern held for models from Alibaba and from Meta, but not its magnitude: the same quantizer cost Meta's model 1.8 points where it cost Alibaba's 0.7. A model's certainty about a token predicts how likely it is to differ from the full model's choice, but not how much that difference affects judged quality, so acceptance rules relying on certainty cannot distinguish errors that matter from errors that don't.

0.65

Where Does Timbre Hide? Localizing Speaker Information in Speech Signals

cs.SD cs.AI ·2026-10-21 ·Yiquan Zhou, Yixuan Wang, Xiaozhe Qi, Xin Gao, Shihao Li
published similarity 0.53
Abstract · accepted by SLT2026

Timbre similarity is a key evaluation target for generative speech systems, including voice conversion, speaker-cloning text-to-speech, and zero-shot speech models. When the generated voice does not match the target speaker, the problem often reflects imperfect separation of speaker-related timbre from content, making it important to identify where timbre information resides and which representations still carry it. Using speaker identity as a measurable proxy, we probe frequency bands, phase/magnitude structure, WORLD source/filter decomposition, voiced/unvoiced regions, self-supervised learning (SSL) layers, EnCodec/HuBERT tokens, and a 45-listener test across Mandarin, English, and Japanese speech/singing data. Speaker information is distributed: 1--8 kHz magnitude and vocal-tract spectral-envelope structure dominate perception, above-8 kHz bands remain machine-discriminative, shallow SSL layers expose compact speaker information, and token leakage depends on tokenizer/layer choice. These findings indicate where generation systems should condition on or transform speaker information, and where content, codec, and token pipelines should audit residual speaker leakage.

0.24