CS Paper Radar

Reportable Shortcuts with RLBWTs

Travis Gagie

cs.DS ·submitted 2026-09-06 13:27 UTC · v3

Abstract

Backward search over a run-length compressed Burrows-Wheeler transform (RLBWT) takes one step per pattern character, even when long stretches of the pattern match material that recurs throughout a highly repetitive collection. We describe shortcuts: for a precomputed set of pairs $(α,β)$ of strings, a backward search that has just matched $β$ and finds $α$ immediately to its left in the pattern can cross $α$ in one step. Since a shortcut is taken only after its pair has been recognized in the pattern, the search returns a report of the shortcuts it took along with the usual BWT interval. Our motivation is pangenome graphs, where such pairs mark the places where a pattern passes through a vertex, so the report gives the path a match takes through the graph. We work out the case of a single linear reference $R$, with $β$ ranging over the length-$\ell$ windows of $R$ starting at multiples of $\ell$, and $α$ over the $2^t \ell$ characters preceding such a window. We show that the search counts correctly, that under stated conditions it crosses a stretch of the pattern matching $m$ consecutive windows of $R$ with $O(\log m + m / 2^τ)$ shortcuts, where $2^τ\ell$ is the longest shortcut, and that the shortcuts can be built efficiently from the RLBWT. On the Pizza&Chili cere collection of 37 genomes, shortcuts make backward search 2.3 to 4.2 times faster on exact patterns of length 200 to 1000, and 1.7 to 2.0 times faster with 1% of the characters substituted, for 86% more space. On 11 gigabases of genomes of seven bacterial species, they make it 1.6 to 1.9 times faster on exact patterns for 32% more space. On whole chromosome 19 haplotypes against 16 to 64 others from the Human Pangenome Reference Consortium, shortcuts in both directions make Boyer-Moore-Li search for maximal exact matches of length at least 1000 about 8 times faster.

Closest papers

Nearest neighbours by meaning across the whole index.

More RLBWT Tricks

cs.DS ·2026-09-30 ·Travis Gagie
theory similarity 0.80
Abstract

Backward search over the run-length compressed Burrows-Wheeler transform (RLBWT) of a pangenome takes one step per character of the pattern, even when long stretches of the pattern match material that occurs many times in the collection. Two-level indexing also indexes a parse of the collection, so that a search can take one step per phrase. We parse at closed syncmers, whose phrases are prefix-free, and add blocks of continuations: when each phrase of a stretch of the pattern is the smallest phrase that follows the previous one anywhere in the collection, the search crosses the whole stretch in one step, on the BWT of a sequence of block symbols. We prove that the three BWTs stay in the same order, so the search still returns exactly the BWT interval of every match, and we bound the extra space by terms that depend on the length of the collection only through the number of distinct closed syncmers. On chromosome 19 haplotypes from the Human Pangenome Reference Consortium, finding the maximal exact matches of length at least 1000 of two haplotypes, on both strands, takes 5.5 to 5.7 seconds with 16, 32 or 64 other individuals, while backward search on a run-length compressed suffix array of the same collection takes 40 to 60 seconds and grows with the collection. We compare this with suffix-tree path decompositions (STPDs): over the parse, an STPD is no faster and needs about twice the memory; over the text it is about 1.7 times faster but needs the whole text in memory, about five times the space; and the published STPD-index, which compresses its text with relative Lempel-Ziv, is 8 to 10 times slower. Keeping a toehold through all three kinds of steps lets the search also list every occurrence of every MEM, for about 2 more seconds.

0.10

Revisiting (Bijective) Burrows-Wheeler Transform Space-Time Trade-Off Constructions

cs.DS ·2026-11-02 ·Dominik Köppl, Tomohiro I
similarity 0.65
Abstract

The bijective Burrows-Wheeler transform (BBWT) replaces the delimiter of the ordinary BWT by a Lyndon factorization and sorts factor conjugates in $ω$-order. It admits quadratic-time constant-workspace algorithms and linear-time linear-space algorithms. We extend the small-space BWT trade-off of Crochemore et al. [JDA,'15] to the BBWT and accelerate both constructions. For $τ\in [3..n]$, let $σ_τ\le\min(σ,τ+1)$ be the maximum construction-batch alphabet size. We construct either transform in $ O\!\left( \frac{n^2}τ \frac{\logτ}{\log\logτ} \right) $ time and $O(τσ_τ)$ additional space, assuming $τσ_τ=O(n)$. This improves the transferred $O((n^2/τ)\logτ)$ bound by a $\log\logτ$ factor. If $σ_τ\in w^{O(1)}$, where $w\inΩ(\log n)$ is the word size, the time improves to $ O\!\left( \frac{n^2}τ + \frac{n\logτ}{\log\logτ} \right). $ For inversion, the same bounds hold with the global alphabet size $σ$ in place of $σ_τ$.

0.21

Towards Optimal Prefix-Free Graph Construction: NP-Hardness and Structural Insights

cs.CC ·2026-09-15 ·Andrej Baláž, Alexandru Popa
similarity 0.61
Abstract · 11 pages, 0 figures

Prefix-free parsing provides an efficient way to construct compressed representations of large and repetitive pangenomes and naturally induces a graph representation known as a prefix-free graph. In this work, we initiate a theoretical study of the problem of constructing prefix-free graphs of minimum size, where the size accounts for both the total length of distinct segment labels and the paths representing the input sequences. We show that selecting an optimal set of trigger words is NP-hard, already when triggers consist of single characters. Using a synchronized-code reduction, we extend this hardness result to every fixed trigger length and further show that the problem remains NP-hard over an alphabet of size three. We then establish a structural connection between prefix-free graphs and de Bruijn graphs. In particular, we show that every compacted de Bruijn graph can be realized as a prefix-free graph and derive a hierarchy relating the sizes of minimum pangenomic graphs, minimum prefix-free graphs, compacted de Bruijn graphs, and de Bruijn graphs. Finally, we give an exact fixed-parameter algorithm running in $O(2^q n)$ time, where $q$ is the number of distinct candidate trigger words and $n$ is the total pangenome length. Our results characterize both the computational limitations and the structural properties of optimizing prefix-free graph representations and provide a theoretical foundation for the design of compact graph representations of repetitive pangenomic data.

0.31

Pangenome Optimization via Elastic Degenerate Strings

cs.DS ·2026-09-22 ·Nicola Rizzo, Sebastian Visan-Draghicescu, Nadia Pisanti, Veli Mäkinen
code/data similarity 0.60
Abstract

An Elastic Degenerate String (EDS, or ED-string) is a sequence of string sets. A pangenome, consisting of variations observed in a population along the genome sequences, can be naturally encoded as an EDS. Pattern matching and comparison problems on pangenome representations such as EDSes have been widely studied in the literature, but optimizing the pangenome properties during its construction has been largely omitted. We fill this gap by showing how methods originally developed for the related problem of founder reconstruction can be adapted to minimize, in linear time, the total cardinality of the EDS sets or the total size of the EDS strings, given suitable multiple alignments representing the input data. We provide an implementation for the minimum-cardinality criterion in a tool mincard, and conduct the first experiments on scalable pangenome optimization via EDSes. The code and experiments are available at https://github.com/algbio/eds.

0.39

Efficient Lempel-Ziv Prediction with the Run-Length BWT

cs.DS cs.LG ·2026-11-01 ·Ohad Avnery
similarity 0.59
Abstract · 21 pages, 1 table

Lempel-Ziv Plurality (LZP) is an algorithm of Kosoy that predicts the next symbol of a string by using its Lempel-Ziv factorization, and makes at most $O(z\log n)$ mistakes on every string of length $n$ with $z$ LZ77 phrases. We give efficient online implementations of LZP and of a new variant of it. Our first implementation maintains a run-length encoded Burrows-Wheeler transform (RLBWT) of the reversed history. It implements LZP, uses $O(z\log z\log^2 n)$ bits, and has worst-case update and prediction times $O(\log z+\log\log n)$ and $O(σ(\log z+\log\log n))$, respectively, where $σ$ is the size of the alphabet. This is an exponential improvement over Kosoy's implementation, whose update time is $O(z\log^7 n)$. We then introduce the Lempel-Ziv Majority (LZM) predictor, which has the same mistake bound and update time, and predicts in $O(1)$ time independently of $σ$. These two predictors accept new symbols at any time, without knowing the alphabet in advance. For an alphabet known in advance, a different data structure gives $O(σ(1+\log z/\log\log n))$ time per symbol, which is constant when $σ=O(1)$ and $z=\mathrm{polylog}(n)$.

0.25

Improved Regular Expression Matching with Simple Backreferences

cs.DS ·2026-09-15 ·Philip Bille, Inge Li Gørtz, Rikke Schjeldrup Jessen
theory similarity 0.58
Abstract

A regular expression with backreferences (rewb) specifies a set of strings formed by characters combined with concatenation, union, star operators, and backreferences. A backreference consists of a capturing group $(\cdot)_i$ and a reference $\backslash i$. The substring matched by the reference must match the substring matched by the corresponding capturing group. Given a rewb $R$ and a string $Q$, the rewb matching problem is to decide whether $Q$ is one of the strings specified by $R$. In full generality, rewb matching is NP-complete, but efficient solutions exist for various subclasses. In the paper, we focus on rewb containing a single capturing group and $k$ references. For this class, Uezato~[CPM 2026] gave an $O((k n^2 m^2)$ time and $O(n^2m^2)$ space algorithm, where $m$ is the length of the regular expression $R$ and $n$ is the length of the string $Q$. For the special case of $k=1$, Nogami and Terauchi~[MFCS 2025] gave an $O(n^2m^2)$ time and $O(n+ m^2)$ space algorithm. On the other hand, Nogami, Nakamura, and Terauchi~[arXiv 2026] gave a conditional lower bound, showing that we cannot solve the problem in $O(n^{2-ε} \mathrm{poly}(m))$ for any $ε> 0$ assuming the orthogonal vector hypothesis. Our main result is a new algorithm that runs in $O(n^2m)$ time and uses $O(nm)$ space. This improves the above results (by a factor of $km$ and $m$, respectively) and the former's space bound (by a factor of $nm$). We also show how to extend our algorithm to handle a slightly more general class of ordered and single-nested rewbs.

0.22

Unconditional Boolean Dynamic Lower Bounds from Disjointness

cs.DS cs.CC ·2026-10-07 ·Young Kun Ko
theory similarity 0.58
Abstract · 48 pages, 3 figures. v2: main result stated for the multiphase problem; appendix extended to further dynamic problems; introduction revised.…

We prove an unconditional $Ω((\log n/\log\log n)^2)$ lower bound on the maximum of the worst-case update time and expected query time for the insert-only version of Patrascu's multiphase Disjointness problem [STOC 2010]. The bound holds in the zero-error cell-probe model with logarithmic word size. Patrascu's reductions transfer it to directed reachability under edge insertions, already on directed acyclic graphs of depth $2$. This improves the $\widetildeΩ(\log^{3/2} n)$ bound of Larsen and Yu [SICOMP 2025] to the quadratic-logarithmic scale previously attained for dynamic range counting and multiphase Inner Product. The bound also transfers to a broad collection of dynamic graph, algebraic, geometric, and string problems. We also obtain a separation between amortized and worst-case time for streaming regular-expression membership. For a simple family of expressions of maximum length $M$ over a polynomial-size alphabet, a deterministic algorithm uses $O(1)$ amortized time per symbol on every input stream, whereas every zero-error cell-probe algorithm with word size $w=Θ(\log M)$ requires $Ω((\log M/\log\log M)^2)$ time on some symbol. The proof adapts Ko's 2.5-round multiphase communication framework [FOCS 2026] from Inner Product to Disjointness. Our main technical observation is that the protocol's verification step allows a one-sided corruption bound to replace the discrepancy property used for Inner Product.

0.24

Optimal Pattern-Avoidance Bounds for Binary Search Trees

cs.DS ·2026-11-15 ·Anita Dürr, Arturo Merino, Karol Węgrzycki, Sorrachai Yingchareonthawornchai
theorycode/datahot direction similarity 0.58
Abstract · 37 pages. Submitted to STOC 2027. Publicly available at https://github.com/sigact-stoc/stoc2027/releases/download/papers-2/906.pdf

Self-adjusting binary search trees (BSTs) perform better than balanced BSTs if the access sequence has additional structure. For example, splay trees serve a sequential access sequence in linear time. Pattern avoidance captures regularity in an access sequence and unifies several classic examples of efficient BST access, such as deque and traversal sequences. A central question is how well a BST algorithm can exploit a pattern-avoiding access sequence. We present an online BST algorithm that serves an access sequence avoiding a permutation pattern $π$ in linear time, matching the information-theoretic lower bound. The algorithm does not need to know that pattern $π$ a priori. Our results answer an open problem asked by [Kozma 2016, Problem 44, strong variant]. Previously, the best known offline bound was optimal only for constant-size patterns [Berendsohn, Kozma, and Opler, STOC'24], and the best known online bound was superlinear [Chalermsook, Pettie, Yingchareonthawornchai, SODA'24] even for a constant-size pattern. As a corollary, our BST algorithm yields an optimal sorting algorithm for a pattern-avoiding sequence that supports online insertions. Previously, an optimal sorting algorithm was achieved using a mergesort-like approach [Opler, FOCS'24] in a batch setting. A key technique is to apply recursive split decomposition [Chalermsook, Chuzhoy, Saranurak, ToC'23] jointly in time and space, along with the Marcus-Tardos proof of the Stanley-Wilf conjecture. We show that the framework can be implemented online using $k$-finger BSTs, which is of independent interest. Finally, this framework also yields a constant-factor-approximation FPT algorithm in the offline setting.

0.41