CS Paper Radar

Linear Network Coding for Sum All-Reduce over Ring Networks

Yi Chen, Zhuoqi Tu, Shenghao Yang

cs.IT ·submitted 2026-11-16 14:46 UTC · v1
publishedtheory

Abstract

Sum All-Reduce is a fundamental collective communication primitive in which each node holds a local vector and every node must recover the componentwise sum of all vectors. In distributed training, this operation is often implemented over a ring topology using a reduce--multicast scheme. We study sum All-Reduce on directed rings via linear network coding, which includes reduce--multicast as a special case. We give a necessary and sufficient feasibility condition in terms of the coding matrices. For a ring with $N\geq 3$ nodes and unit-capacity links, where each node holds a vector containing $K$ symbols, we prove that the number of time units $T$ required by any feasible linear code satisfies $T\ge\max\bigl\{K+N-2,\ \lceil\frac{2(N-1)}{N}K\rceil\bigr\}$, giving a rate $K/T$ of at most $N/(2(N-1))$. The bound is tight over every field. When $K\ge N$, an interleaved reduce--multicast code achieves the bound; when $K<N$ and $K$ divides $N$, a reduce--partial-multicast code does. When $K<N$ and $K$ does not divide $N$, no coordinate-wise code---in particular, no reduce--multicast code---achieves the bound, while an explicit linear network code does.

Closest papers

Nearest neighbours by meaning across the whole index.

Communication-Efficient Distributed Training via Ring-Based Coded Approximate All-Reduce

cs.IT ·2026-09-20 ·Sifat Munim, Aditya Ramamoorthy
similarity 0.70
Abstract · 21 pages, 1 figure

Ring All-Reduce is a widely used protocol for aggregating gradients across workers in large-scale distributed training. For a system with $N$ workers, its normalized per-worker communication is $2(N-1)/N$. In this work we present a communication-efficient Ring All-Reduce (CERAR) protocol for ``approximate'' gradient aggregation. CERAR partitions each local gradient into $c$ components and relies on linear encoding and decoding operations. It performs $L=c+N-2$ communication rounds over an $N$-worker ring, yielding normalized communication rate $1+(N-2)/c$ and normalized storage rate $1+N/c$. We present an explicit Vandermonde matrix based construction whose approximation error can be made arbitrarily small, with communication and storage rates approaching one as $c$ increases. However, this comes at the cost of ill-conditioned encoding matrices. We therefore give an alternate construction whose error is $O(ε)$, while the relevant condition numbers are $O(ε^{-r_\star})$, where $r_\star=\lceil c/N\rceil-1$. This motivates a condition-number-constrained optimization formulation for obtaining numerically stable practical designs. Our experiments show that, compared with the production-standard NCCL on NVIDIA A100 PCIe GPU clusters with low-bandwidth PCIe interconnect, CERAR reduces raw All-Reduce time by up to 47\%. For training a 1.4-billion-parameter Pythia model, it reduces training time by about 17\% for the same number of iterations while achieving essentially identical validation loss. On GPU clusters with high-bandwidth NVLink interconnects, CERAR performs worse in the All-Reduce phase and slightly worse in overall training time. However, the gap narrows for larger model sizes, suggesting that further gains may be possible with more optimized CERAR implementations.

0.29

Deterministic Distributed Linearly Separable Computation via Nested Cyclic Codes

cs.IT ·2026-10-13 ·Anu Mathew, M. Nikhil Krishnan
similarity 0.60
Abstract · 13 pages, 1 figure

We study the $(K,N,S,M,K_c)$ distributed linearly separable computation (DLSC) setting, where a master employs $N$ workers with a per-worker computation load of $M$ to recover $K_c$ deterministic linear combinations of $K$ messages in the presence of up to $S$ stragglers. We propose a DLSC framework based on nested cyclic codes that achieves $M=\frac{K}{N}(S+K_c)$. For sufficiently large $K$, we establish the matching converse $M\geq\frac{K}{N}(S+K_c)$, proving the asymptotic computation load optimality of the proposed framework. By instantiating cyclic locally recoverable codes (LRCs) as base codes in the framework, the master can recover the desired $K_c$ linear combinations by contacting fewer than $N-S$ workers under many straggler scenarios. This essentially softens the straggler threshold without increasing the computation load or communication cost. Finally, we extend the construction to reduce the per-worker communication load by a factor of $m$, while achieving an asymptotically optimal computation load of $M=\frac{K}{N}(S+mK_c)$.

0.20

Network Coding Can Beat Routing in Undirected Multiple-Unicast Networks

cs.IT ·2026-10-07 ·Xindan Zhang, Baochun Li, Zongpeng Li
theoryhighly cited similarity 0.60
Abstract · 17 pages

Network coding lets the nodes of a network combine messages rather than merely forward them. In undirected networks, where the two directions of an edge share its capacity, Li and Li conjectured in 2004 that coding offers no advantage over fractional routing; the conjecture has been confirmed for many special classes but never settled in general. In this paper, we disprove it by constructing a finite connected simple undirected network with unit shared edge capacities, maximum degree three, and distinct leaf terminals, on which a binary linear block code achieves a common rate strictly above the maximum fractional routing rate. The construction turns a short completion-time code into a reversible circuit whose registers all carry independent messages, and a temporal metric on its wires yields the strict routing bound. The underlying integer-coefficient construction works over every finite field and every nontrivial finite abelian group; for deterministic fixed-schedule codes the common-rate supremum is one and the closure of the rate region is the unit cube. Amplifying the gap with a degree-preserving tensor construction and an even subdivision gives, for every $0<β<1$, an unbounded family of connected, simple, subcubic, bipartite networks with girth at least $n^β$ on which coding approaches rate one while routing is bounded by $K_β/(\log n)^c$, with one positive exponent $c$ independent of $β$. The finite counterexample and the family theorem are formalized in Lean.

0.64

When Multiple-Unicast Feasibility Is Equivalent to Single-Sink Routability

cs.IT ·2026-10-24 ·Sirui Liu, Zongpeng Li, Baochun Li
theory similarity 0.59
Abstract · 14 pages, 3 figures. Revised exposition and proof presentation

In communication networks, network coding allows intermediate nodes to combine messages, achieving transmission rates unattainable by routing alone in many multicast settings. In multiple unicast, where each source sends a message only to its prescribed destination, a central question is under what conditions fractional routing suffices to attain the transmission rates supported by network coding. We prove that, for any given collection of multiple-unicast demands in an undirected network, if the maximum flow between every pair of distinct destinations is at least the total demand rate, then the following are equivalent in feasibility: multiple-unicast coding, multiple-unicast fractional routing, and simultaneous fractional routing of all source rates to any fixed destination. Verifying this pairwise maximum-flow condition and, when it holds, deciding feasibility of the given multiple-unicast demands require at most as many maximum-flow computations as there are destinations. As a direct corollary, we prove the Langberg--Médard conjecture: if a directed network supports coded multicast of independent source messages at a common rate to every destination, then removing arc directions without changing capacities permits simultaneous fractional routing at that rate for all prescribed source--destination pairs.

0.25

Minimality of Binary Codes From Weight Layers and Graph Cut Spaces

cs.IT ·2026-10-26 ·Xianhong Xie, Mengni Wang, Tingting Wu
similarity 0.58
Abstract

Minimal binary linear codes play an important role in secret-sharing schemes and in the structural study of linear codes. We study the minimality of three classes of known constructions: codes from a single complete Hamming-weight layer, codes from arbitrary unions of such layers and graph cut-space codes. For the previously studied single-layer codes C, we give a complete minimality classification: C is minimal if and only if t is even. For t = 4, we determine the minimum and maximum weights, and show that the family strictly violates the binary Ashikhmin Barg sufficient condition for every k. We also calculate the minimum distance of the dual code. For arbitrary unions of complete Hamming-weight layers, we obtain an if-and-only-if classification of minimality and uniform criteria for evenness, self-orthogonality, the doubly even property and hull dimension. We further analyze the family for odd k. Finally, using the classical bond characterization of cuts, we translate minimality of graph cut space codes into a graph-theoretic condition: a nonempty simple graph has a minimal cut code precisely when it has one edge-containing component and that component is complete. We also give a graph-function realization of the known binary tetrahedron code and explicitly identify the resulting code equivalence.

0.24

Optimal-Access Cooperative MSR Codes: Parity-Check Matrix Construction And a Unified Transformation

cs.IT ·2026-09-06 ·Yaqian Zhang, Jingke Xu, Ya-Feng Liu
theory similarity 0.58
Abstract

Cooperative MSR codes are a kind of storage codes which enable optimal-bandwidth repair of any $h\geq2$ node erasures in a cooperative way, while retaining the minimum storage as an $[n,k]$ MDS code. Each code coordinate (node) is assumed to store an array of $\ell$ symbols, where $\ell$ is termed as sub-packetization. To address the disk IO (input/output) capability, a cooperative MSR code is said to have optimal-access property, if during node repair, the amount of data accessed at each helper node meets a lower bound on this quantity. In this paper, we focus on reducing the sub-packetization level of optimal-access cooperative MSR codes. We propose new constructions of optimal-access cooperative MSR codes through two methods. At first, we propose a direct explicit construction by designing its parity-check matrix. Such parity-check matrix is built by repeatedly employing two crucial parity-check matrices as building blocks. Secondly, we propose a generic transformation framework. Starting from an arbitrary $[n+d-k,d]$ MDS scalar code, one can derive a final cooperative MSR code by systematically applying two basic transformations. Both approaches yield $(n,k,\ell=δ^m)$ optimal-access cooperative MSR codes with $δ=d-k+h$ and $m=\binom{n}{h}-\lfloor\frac{n}δ\rfloor(\binomδ{h}-1)$. Compared with the state of the art (with $\ell=δ^{\binom{n}{h}}$), the derived codes can reduce the sub-packetization $\ell$ by a fraction of $1/δ^{\lfloor\frac{n}δ\rfloor(\binomδ{h}-1)}$, where $δ=d-k+h$. Moreover, we also show that some previous code structures of optimal-access cooperative MSR codes and optimal-access MSR codes with $h=1$ are included as special cases of our transformation construction. At last, we note that all of the constructions are built over a finite field of linear size $\geq n+d-k$.

0.30

The Multiple Unicast Conjecture is False

cs.IT cs.CCcs.DMcs.DS ·2026-10-07 ·Mark Braverman, Zhongtian He
theoryhighly cited similarity 0.57
Abstract

The undirected multiple-unicast conjecture [LL04] asserts that network coding offers no throughput advantage over multicommodity flow. We refute this conjecture by constructing a deterministic linear network code over $\mathbb{F}_9$ on a 182-vertex bipartite subgraph of the point-line incidence graph of $\mathrm{PG}(2,9)$. The construction supports $157$ independent unicast sessions at common coding rate at least $1$, while every fractional multicommodity flow has common rate at most $147/157$. By the amplification theorem of~[BGS17], this yields a family of undirected multiple-unicast instances with coding gap $Ω((\log n)^\varepsilon)$ for some $\varepsilon>0$. We also introduce a nondeterministic model of network coding based on locally verifiable certificates, which guides our construction and may be of independent interest. Building on the high-girth graph and error-correcting code framework of [BH25], we use GPT-6 to find a nondeterministic counterexample based on a new choice of Reed--Solomon local codes, and then convert this example into a causal code using an edge orientation and local search.

0.38

Convolutional Codes from Cyclic Codes with Guaranteed Free and Local Minimum Distances

cs.IT ·2026-09-08 ·Khaled Abdel-Ghaffar, Daniel J. Costello, Juane Li, Shu Lin
similarity 0.57
Abstract

This paper presents an algebraic method to construct convolutional codes with guaranteed \emph{free and local minimum distances} without limit based on cyclic codes of odd lengths. The constructions are simple but effective, and no computer search is needed. For any two positive integers $r$ and $t$ with $1 \leq r < t$, a rate-$r/t$ convolutional code $\mathcal{C}_{\text{convol}}$ can be constructed by using a chain of $r$ cyclic codes $\mathcal{C}_0, \mathcal{C}_1, \ldots, \mathcal{C}_{r-1}$ of the same length $n$ which satisfy the inclusion condition, $\mathcal{C}_0 \supset \mathcal{C}_1 \supset \ldots \supset \mathcal{C}_{r-1}$. Such a convolutional code $\mathcal{C}_{\text{convol}}$ is composed of a \emph{semi-infinite chain of identical local codes} confined in a diagonal band of width $n$. Each local code $\mathcal{C}_{\text{local}}$ of $\mathcal{C}_{\text{convol}}$ is formed from the $r$ cyclic codes in the code chain and is a specially localized subcode of the \emph{mother code} $\mathcal{C}_0$ in the code chain. The minimum distance $d_{\text{local}}$ of each local code of $\mathcal{C}_{\text{convol}}$ is lower bounded by the minimum distance $d_0$ of the mother code $\mathcal{C}_0$ in the code chain. The local structure of $\mathcal{C}_{\text{convol}}$ allows it to be decoded based on a designed parity-check matrix of the mother code $\mathcal{C}_0$ using a sliding window decoding scheme.

0.08