CS Paper Radar

What reviewers call a Long Method is a length: A pre-registered test of coupling divergence as a god-unit signal, and what it settled instead

Daniel Sagenschneider

cs.SE ·submitted 2026-11-01 06:39 UTC · v1

Abstract

A companion study found that a complexity outlier rule shows where complexity concentrates but cannot tell a god unit from a complex algorithm. This paper pre-registered a test of one way to tell them apart: weight a unit's complexity by its coupling divergence, the data it hands on minus the data it receives, so that a leaf is set aside. Coupling was measured by resolving every call in 520 MLCQ repositories, 500 fresh Java repositories and 73 Qualitas Corpus systems. The claim fails: the primary, the pooled MCC of the weighted form minus complexity alone against MLCQ's human Blob, is -0.041 [-0.130, +0.054], no difference resolved and underpowered, and the resolved readings go against it. What the data settle is at method level. What MLCQ's reviewers call a Long Method is an absolute length: a constant of 26 lines or more, chosen on the explore half, reaches MCC 0.777 on the confirm half, +0.080 [+0.016, +0.145] over the best per-repository form; every reviewer with ten or more positives, on their non-leaf reviews, has a best cutoff between 17 and 34 lines; and nothing measured inside a method, its calls, variables, data flow or the cost of cutting it, refines the constant. The constant is a limit of human reading. The next form of the construct is the cost of bringing a long method under it, which is where the construct meets architecture.

Comment
6 pages, 1 table. Ancillary files: the full registered report and the analysis data. Also deposited at Zenodo, doi:10.5281/zenodo.23749773

Closest papers

Nearest neighbours by meaning across the whole index.

Complexity concentration as a god-class and god-method detector: a pre-registered test of a scale-free outlier rule against tool definitions and human labels

cs.SE ·2026-10-23 ·Daniel Sagenschneider
unusual similarity 0.68
Abstract · 16 pages, 1 figure, 6 tables. Pre-registered study; ancillary files contain the run data, analysis outputs and readings. Archived at doi:10.…

God-class and god-method detectors mark concentrated complexity with a raw-scale threshold, a constant fitted on a corpus and re-fitted when the population moves. This paper pre-registers a test of a rule with no raw-scale constant: a unit is a god unit when its complexity is a robust outlier for its own codebase, a modified z-score of 3.5 or more. On 445 repositories in ten languages, 171 of them Java, and the 492 Java repositories of MLCQ, the rule is scored against PMD's GodClass, MLCQ's reviewers and two further definitions, with signal and cutoff chosen on half the repositories and read on the rest. LLM-written code may gather its complexity differently, the case for a detector that needs no re-fitting; the first question is whether one can match the scale of existing human-written code. The verdict is inconclusive on both populations, by different arms. The rule ranks PMD's god classes from the per-file maximum of cognitive complexity at per-repository AUCs of 0.916 and 0.908. On the main run it is, at its default, non-inferior within the margin to a cutoff tuned on the same population on recovery, and flags about twice as many non-god classes; on MLCQ it flags about as many and falls short of the margin on pooled recovery. Pooled, the raw value ranks god classes better than the per-repository score, and a cutoff carried between the populations lost almost nothing: both are human-written code on one scale, so the test is of matching that scale, not of surviving a shift. The rule's extra flags are small classes, just over half holding a method at PMD's own method-level threshold; its misses are large classes whose complexity is spread over many methods. Two readers of the source did not agree on what the extra flags are. Concentration finds where the complexity is; whether that place is a god unit is a question about coupling, which a registered follow-on study takes up.

0.53

Truncation Samplers Are Nested: Comparing Decoding Rules Is a Measurement Problem

cs.LG cs.CV ·2026-10-11 ·Rasul Khanbayov, Hasan Kurban
benchmarkcode/data similarity 0.51
Abstract

Settling one comparison between two truncation samplers has cost six thousand A100-hours, and nothing says in advance how far apart two samplers can be. We observe that almost every deployed truncation rule keeps a prefix of the probability-sorted vocabulary, so any two such rules are nested at every decoding step. Nesting yields a closed-form per-step distance and a coupling under which both samplers emit identical text until they first disagree. Accumulated over a generation, this gives a separation certificate that bounds the difference between two samplers on any bounded outcome, is computable from logs a comparison already produces, and equals their sequence-level total variation distance exactly when neither rule ever overtakes the other. Two consequences follow without measurement: exactness is fragile, and certifying a small difference requires the two retained masses to agree to very high precision at every step. On six open-weight models over four math and science benchmarks, the certificate for every hyperparameter grid we tested is at least 0.998, so a bound alone rarely settles a comparison, although two members of one family can be far closer. What remains useful is the coupling, which gives an unbiased paired estimator that cuts variance $1.2$-$3.8\times$ against prompt-level pairing. Read against benchmark sizes at the default pair's variance and our point estimate of its between-prompt component, resolving a one-point difference needs 8 to 32 samples per prompt on GSM8K, MATH-500 and GPQA-Diamond and about 210 on AIME 2025; at that component's upper confidence bound GPQA-Diamond needs more than 32 and AIME 2025 cannot be resolved at any sample count. Choosing a truncation sampler is a measurement problem at least as much as a design one. The code is available at https://github.com/KurbanIntelligenceLab/truncation-samplers.

0.38

LongForge: Measuring and Improving Long-Context Instruction Data via Evidence-Assertion Reasoning Graphs

cs.AI ·2026-10-16 ·Ziyang Chen, Junlong Jia, Xing Wu, Tinghao Yu, Songlin Hu
similarity 0.50
Abstract · Work in progress

Existing quality control for long-context instruction-tuning data is a one-shot judgment inside the generation pipeline that leaves only a scalar, revealing neither which document parts an answer uses nor how many reasoning steps it writes down; the data can thus be neither audited nor controlled. We formalize data quality as a property of the alignment structure among document, question, and answer, and build LongForge, which pairs an analyzer that measures long-context data with a synthesizer that generates it. The analyzer reverse-reconstructs each sample into an evidence-assertion reasoning graph and programmatically computes four dimensions on it: breadth, depth, correctness, and completeness. Every reading traces back to specific evidence spans and reasoning steps, making data auditable. Measuring seven mainstream datasets reveals two previously unquantified defects: breadth barely grows with document length, and depth is generally low, the two being nearly orthogonal. The synthesizer then embeds the analyzer into generation, gating on correctness and completeness and deferring breadth and depth to a training-time mixture over measured values, which makes the training distribution controllable. On two backbones, its two mixtures, LongForge-1 and LongForge-N, outperform the strongest baseline by 2.2-4.7 points on average over six long-context benchmarks, and in a controlled comparison that holds the gate and the candidate pool fixed, raising breadth and depth alone is worth a further 1.16 points, making the structural distribution of training data a lever on downstream quality rather than only a description of it.

0.28

Beyond Typical Strings: Sampling the Complexity Landscape

cs.IT ·2026-10-30 ·Abdurrahman Hadi Erturk, Joshua Sharkey, Nora S Martin, Kamal Dingle, Ard A Louis
similarity 0.50
Abstract

Uniform random sampling of strings is overwhelmingly dominated by high-complexity strings, leaving the much smaller set of low- and intermediate-complexity strings largely unexplored. Here, we adapt the nested neutral set estimator (NNSE), a Monte Carlo rare-event sampling method, to estimate the distribution P(K) of complexity values for binary strings of fixed length. NNSE allows us to explore the full complexity landscape, including exponentially rare low-complexity regions that are inaccessible to direct sampling, for string lengths up to several hundred. We apply the method to several widely used complexity measures: Lempel-Ziv 76 (LZ76), Lempel-Ziv 78 (LZ78), context tree weighting (CTW), the block decomposition method (BDM), and block entropy. We further introduce uniform complexity weighting (UCW), which gives equal weight to different complexity levels rather than to individual strings. Using NNSE to perform UCW sampling allows us to compare complexity measures across their full range in a dataset-independent way. This reveals differences in their behavior for low- and intermediate-complexity strings that are practically inaccessible to conventional uniform sampling.

0.56

Capability-Gated Conformance Testing of Quantum Error-Correction Decoder Libraries

cs.AR quant-ph ·2026-09-07 ·Jiachen Shen, Hui Zhong
similarity 0.47
Abstract

A quantum error correction decoder is a library other people's results depend on, judged in one dominant way. Sample errors, decode, and count wrong logical observables. We ask what else can be checked there. Our conformance contract needs no oracle. One check asks that a returned correction explain the syndrome in the caller's index space. The other hands a decoder one instance under two presentations differing only in bookkeeping, where two feasible corrections of different weight prove the heavier is not minimum-weight. Verdicts are gated on what each library declares, so a firing contradicts a published guarantee. Nine configurations from five public libraries give three results. Documentation answers 4 of 54 capability questions. Bounded-distance correctness, the property callers most depend on, has a direct declaration yield of 0.0%, though its hypotheses hold in 62.1% of cases. Presentation sensitivity is real but shallow. One solver moved to a 26% heavier correction under a different numbering, which reaches the logical class at most once in twenty thousand shots. Established evaluation misses corruptions that preserve logical parity, while one summation over the caller's weights catches every one we injected. All 639 certificates ship as bundles a standalone verifier re-derives from first principles.

0.29

What extraction cannot slim, and whether function injection can: a pre-registered study of 580 Java repositories

cs.SE ·2026-11-07 ·Daniel Sagenschneider
publishedcode/dataunusual similarity 0.47
Abstract · 19 pages, 1 figure, 10 tables. Pre-registered; every number regenerated by script. Replication package in the ancillary files and at https:/…

A method of 26 code lines or more reads as a Long Method, and Extract Method is the usual fix. Some long methods have no single-exit fix: a return cannot leave an extracted piece, a piece hands back one value, and some methods are knotted so that every cut is blocked. Function injection, as the OfficeFloor framework does it, cuts differently: each piece becomes a function the framework injects and sequences, an early return ends a function, and a function hands on several values. The framework's author pre-registered a test of whether it takes apart what extraction cannot. A stratified random sample of 580 open-source Java repositories, each pinned before AI coding assistants were in general use, was scanned by released tools whose cost models were fixed before the sample was drawn: 3,026,902 methods, 168,466 of them long. Single-exit extraction cannot slim 22.8% of the long methods. The registered hypothesis, that at least a quarter of those stuck methods are ones only function injection takes apart, is refuted: 22.2% (95% CI 20.3 to 24.3). Nearly half of the stuck methods are early-return methods the plan counts as conventionally fixable, and the split takes three in four of those apart as they stand. Where extraction would do, the split chooses injection and costs less for 29.0% against extraction wrapped as one function and 17.2% against plain extraction, almost always as a straight sequence of functions. An exploratory reading of the quarter of the stuck methods that neither takes apart finds a third of it tables and literals, long for their data rather than their logic, and most of the rest blocked by a jump inside a loop, by a nested statement too large to cut, or by values too many for any cut. Three registered studies are proposed: the classes as rules in the tools, a refinement per class, and the same measures on code written with AI assistants.

0.56

Component-Weighted Centroid Search for Exact Incremental BPE

cs.DS cs.LG ·2026-09-30 ·Harshit Verma, Rex Ying
published similarity 0.47
Abstract · AXIOM: Foundations of Efficient Deep Learning workshop at NeurIPS 2026. 4 pages, 2 figures, appendix

Exact incremental tokenizers keep the tokenization of a growing text up to date, so a long prompt or agent trace is not retokenized each time it grows. Jiang and Gong's algorithm does this for byte-pair encoding (BPE) in $O(\log^2 t)$ worst-case time per byte, where $t$ is the length of the longest token. Its search passes through $O(\log t)$ nested subproblems and runs a binary search over the possible branches at each one. We weight each branch by the size of the subproblem it leads to. Each step then costs about one probe for every halving of the remaining candidates, and a whole update costs $O(\log t)$, as much as one binary search over all candidates. Output and asymptotic memory are unchanged. We also construct valid BPE vocabularies on which the original search provably needs $Θ(\log^2 t)$ probes and ours $Θ(\log t)$, and a Rust implementation reproduces both counts exactly. On six real vocabularies the two searches run at about the same speed on ordinary text, while on long runs of spaces or hyphens the weighted search is up to 2.5$\times$ faster.

0.39

Where Induction Runs Out: Description-Length Difficulty and the Memorisation Gap in Integer-Sequence Benchmarks

cs.LG cs.SC ·2026-08-29 ·Sabilashan Ganeshan
benchmarkcode/data similarity 0.47
Abstract · 14 pages, 6 figures. Code and data: https://github.com/sabilashang/where-induction-runs-out. Archived: doi:10.5281/zenodo.21830331

Integer sequences from the On-Line Encyclopedia of Integer Sequences (OEIS) are increasingly used to benchmark mathematical reasoning in language models. We ask what such benchmarks actually measure, using an exactly computable reference learner: two-part minimum description length (MDL) over the class of P-recursive (holonomic) recurrences, evaluated on every prefix of a sequence as terms arrive. Three findings follow. First, MDL difficulty is a parameter count. The discovery point nd, the first prefix length at which a symbolic hypothesis beats verbatim storage, is predicted almost exactly by a combinatorial identifiability bound on the selected operator's order and degree. It is invariant to term magnitude: scaling Fibonacci over twelve orders of magnitude leaves nd unchanged, because a hypothesis must encode its own initial conditions and the magnitude cancels. Second, at scale the learner exhibits a regime our curated corpus could not produce even once: across 20,000 OEIS sequences, 89.98% of those that fit a recurrence on some prefix fit none at full length. We call this the wilderness -- induction acquires a theory, loses it, and never recovers. Third, evaluating three language models on sequences stratified by these MDL regimes refuted our pre-registered hypothesis: models do not confabulate where MDL reports no theory, but hedge appropriately. Confident errors are inverted, concentrating on the easy stratum, where apparent competence tracks recognition of the sequence rather than induction of its rule. OEIS-derived benchmarks therefore substantially measure memorisation, and MDL supplies a cheap, contamination-free difficulty signal they currently lack. Code and data are released.

0.37