CS Paper Radar

Dynamic Coded Distributed Computing with Joint File Placement and Reduce Function Reassignment

Chenglin Li, Minquan Cheng, Youlong Wu

cs.IT ·submitted 2026-09-30 09:25 UTC · v5
theory

Abstract

Coded distributed computing (CDC) mitigates the Shuffle communication bottleneck through coded multicast, but existing schemes with predetermined worker populations do not directly adapt to worker arrivals and disconnections. To address these dynamics, we propose a dynamic CDC scheme that preserves surviving workers' caches and Reduce assignments while exploiting the storage brought by arriving clusters. The model accommodates general MapReduce tasks under a placement that is partly fixed and partly new. The scheme uses Cartesian products of cluster-wise placement delivery arrays (PDAs) for file placement. A delivery-aware greedy algorithm reassigns abandoned Reduce functions, and a Communication Array allows workers with no Reduce function to serve as coded transmitters. For arbitrary feasible disconnections, the scheme's exact communication load is derived. A converse bound under one-shot communication shows that, in the absence of disconnections, the scheme lies within a factor of 2 of the lower bound under these assumptions when new clusters arrive, and within a factor of 4 otherwise.

Closest papers

Nearest neighbours by meaning across the whole index.

Latency-Optimal Geo-Distributed Storage over Structured Networks

cs.IT eess.SY ·2026-09-04 ·Madhura Pathegama, Viveck Cadambe
theory similarity 0.61
Abstract · Accepted to the 2026 IEEE Information Theory Workshop (ITW 2026)

We study latency-optimal file assignment in geo-distributed storage systems modeled as weighted graphs, where edge weights represent communication delays and each node stores one (possibly coded) file. Our goal is to minimize the average time required to retrieve an original file, taken uniformly over all nodes and files. We show that for every fixed number of files $k \geq 3$, computing a latency-minimizing assignment is NP-hard via a reduction from the domatic number problem. On the positive side, we identify natural network topologies that admit uncoded, structured optimal assignments in which, for every node, one can choose its $k$ closest nodes, including itself, so that they store distinct original files. We prove that every weighted tree, certain weighted cycles, and unit-weight graphs with sufficiently large minimum degree admit such assignments. For these graph classes, we provide efficient algorithms to construct latency-optimal file assignments.

0.33

The Exact Optimal Delivery Scheme for the Cyclically $L$-Separated Placement in Multi-Access Coded Caching

cs.IT ·2026-11-16 ·Kai Wan, Huimei Wei, Minquan Cheng, Giuseppe Caire
similarity 0.61
Abstract

This paper investigates the $(K, L, M, N)$ multi-access coded caching (MACC) system introduced by Hachem et al. The system consists of a server storing $N$ equal-sized files, $K$ cache nodes, each with a storage capacity of $M$ files, and $K$ users, each accessing $L$ consecutive cache nodes in a cyclic wrap-around topology. The server transmits coded information to the users via an error-free capacitated broadcast channel, such that each user obtains its own requested file as a combination of the server transmission and the accessible caches' content. Our objective is to minimize the worst-case transmission load over all possible user demands. First, for the cyclically $L$-separated placement, we derive a new converse bound using the acyclic index coding bound. By carefully counting all valid cache supports, the proposed converse improves upon an existing bound for the same placement. We then develop a linear coded caching scheme based on a novel determinant-based interference alignment strategy, which achieves the new converse bound exactly. Exploiting the determinant structure of a random generating matrix (each of whose columns corresponds to one user), this scheme simultaneously aligns interference across all users into low-dimensional subspaces, while preserving the decodability of their desired packets. Furthermore, by averaging acyclic index coding bounds over cyclic user permutations, we derive a general converse bound applicable to arbitrary uncoded cache placements. By combining the proposed scheme with memory sharing, we demonstrate that the achievable transmission load is within a factor of two of the global bound under uncoded placement. Finally, theoretical and numerical comparisons demonstrate that the proposed scheme generally achieves a strictly lower load than existing schemes.

0.21

Cache-Aided Linear Function Computation

cs.IT ·2026-10-14 ·Xiaohong Cai, Ruze Zhang, Laigang Guo, Raymond W. Yeung
theory similarity 0.58
Abstract · 11 pages, 2 figures. A shorter version has been submitted to the 2027 IEEE Information Theory Workshop (ITW)

We study cache-aided linear computation in distributed systems where each source generates and stores its own data. An external receiver requests a linear combination of these data, and the sources use information cached before the request is known to reduce the communication needed to recover the requested result. In our model, each of the $N$ sources stores its own independent file over a finite field and has an additional cache of normalized size $M$. The receiver must recover any requested linear combination with zero error from the messages sent by the sources over separate noiseless links. We establish a lower bound on the worst-case communication rate over any finite field by analyzing several requests under the same cache placement. This bound determines the optimal rate when no additional cache is available. For binary files, we propose parity-aided placement (PAP), a coded-placement scheme that combines cached parities with uncoded symbols. For every $N\geq3$, we characterize its communication rate for each request and optimize its placement parameters to obtain a closed-form worst-case rate. The resulting achievable rate meets the lower bound for $0\leq M\leq1/(2N-1)$ and $M\geq(N-1)/N$, establishing the optimal memory--rate tradeoff in these two ranges.

0.22

Hotplug Coded Caching: Uncoded versus Row-Wise Coded Placements

cs.IT eess.SP ·2026-11-02 ·Daniela Tuninetti
similarity 0.56
Abstract · Submitted to ICC 2027

In the hotplug coded caching model, $K$ users fill their caches of size $M$ as a function of $N$ files, but only an unknown subset of $A\leq K$ users is active in the delivery phase. In all regimes where the optimal hotplug tradeoff is known, it depends only on $A$, not on $K$; whether this holds in general is open. We show that: (i) Under uncoded placement, YMA delivery with MAN placement, where offline users are assigned the demands of active users, is optimal. Hence, offline users cause a strict load penalty at intermediate memory sizes, while the MAN placement is optimal for every number of active users; (ii) Under row-wise placement (RWP), where each cache is a function of the user's own row of each file, every classical scheme can be embedded in a Reed-Solomon code. The resulting hotplug scheme has the tradeoff of the classical model with $A$ users, for every $A\leq K$: there is no load penalty, but the placement depends on $K$. We then derive a converse for RWP, which shows that: (a) for $2\leq N\leq A$, the scheme of Gomez-Vilardebo (GV) is RWP optimal for $AM\leq N/(N-1)$, which settles the RWP tradeoff for $N=2$; (b) YMA is RWP optimal at $AM=N$; and (c) for $N=A=3$, GV is RWP optimal for $M\leq 1$. Hence, the Reed-Solomon embeddings of these schemes are optimal among embedded RWP schemes.

0.45

Replication-Aware Placement of Functions and Data in the Edge-Cloud Continuum

cs.DC ·2026-09-16 ·Dario d'Abate, Matteo Cenzato, Matteo Briscini, Arianna Dragoni, Alessandro Margara
similarity 0.55
Abstract · Presented at WSCC Workshop @ Euro-Par 2026

Function-as-a-Service (FaaS) has emerged as the prominent programming model for the edge-cloud continuum. FaaS inherently decouples stateless functions from their persistent state. We study how to jointly schedule functions and place data to minimize client latency, considering data replication under heterogeneous consistency requirements. We introduce a Binary Linear Programming (BLP) model to compute optimal placements, establishing a rigorous theoretical baseline. Since the BLP scales cubically with the infrastructure nodes, we propose a topology-aware greedy heuristic that efficiently approximates the optimal solution. Our evaluation shows that the heuristic achieves near-optimal placement quality at a fraction of the computational cost, making it suitable for periodic system reconfigurations.

0.26

BIER-DC: Stateless Multicast for In-Orbit Space Data Centers in LEO Constellations

cs.NI ·2026-10-14 ·Mostafa Abdollahi, Wenjun Yang, Kousar Malekinasab, Jianping Pan
published similarity 0.55
Abstract · accepted by IEEE ICNP 2026

Emerging data-intensive space applications generate large volumes of data that must be processed, stored, replicated, and delivered efficiently. However, relying on terrestrial cloud regions can expose satellite data to high delivery latency and loss of timeliness due to intermittent satellite-to-ground visibility and limited access to ground infrastructure. Space Data Centers (SDCs), which organize collaborative satellites into logical compute and storage clusters, offer a promising approach for in-orbit data management and service continuity. In this paper, we first evaluate how satellite clustering with different sizes affects satellite visibility and gateway outage. We then propose BIER-DC, a stateless one-to-many communication protocol tailored for satellite-based SDCs. Unlike terrestrial multicast protocols designed for stable data center fabrics, BIER-DC decouples inter-SDC forwarding from intra-SDC delivery and encodes compact cluster-level forwarding information in packet headers. Comparative evaluations against BIER, BIER-TE, and Yeti show that BIER-DC significantly reduces packet-header length and byte-hop overhead, improving effective goodput and scalability for next-generation LEO space data centers.

0.38

TCDA: Tunable Coded Distributed Arrays for Data Availability Sampling

cs.DC cs.NI ·2026-10-15 ·Duc A. Tran, Dung Truong, Duy Le
similarity 0.53
Abstract

Data availability sampling (DAS) lets blockchain nodes verify a block's data availability by downloading only a few random pieces of it. Its network layer must disseminate each block in time for validators to attest (PUT), store it, and serve any piece on request (GET). A trustless DAS must sustain any number of Byzantine peers. The only known design with this guarantee, Robust Distributed Arrays (RDA), achieves it by full replication: each group of nodes responsible for part of a block stores that part in full, so a single good node per group suffices. The price to pay is a PUT load on the critical path to finality that grows with the network and cannot be tuned, hence the data may not reach validators before the attestation deadline. Coding is the natural remedy, but it was not known when a coded layer remains robust against an unbounded adversary, what it costs, or what it gains. We propose TCDA, a coded layer that is provably robust given enough good nodes, wherever they lie, with a tunable knob: how much of its assigned data each node keeps. Turning the knob trades PUT and storage costs against GET cost. It can be set per data type and adjusted as the good-node population changes. At Ethereum's parameters, our evaluation for a 32 MiB block at the recommended uplink shows that RDA misses the attestation deadline at any network size, whereas TCDA can be tuned to meet it from 868 good nodes up to the 40,000 tested, storing 1.7 to 32 times less per node than replication, at the price of 2.5 to 7.7 round-trips per sample instead of about one.

0.20

Deterministic Distributed Linearly Separable Computation via Nested Cyclic Codes

cs.IT ·2026-10-13 ·Anu Mathew, M. Nikhil Krishnan
similarity 0.53
Abstract · 13 pages, 1 figure

We study the $(K,N,S,M,K_c)$ distributed linearly separable computation (DLSC) setting, where a master employs $N$ workers with a per-worker computation load of $M$ to recover $K_c$ deterministic linear combinations of $K$ messages in the presence of up to $S$ stragglers. We propose a DLSC framework based on nested cyclic codes that achieves $M=\frac{K}{N}(S+K_c)$. For sufficiently large $K$, we establish the matching converse $M\geq\frac{K}{N}(S+K_c)$, proving the asymptotic computation load optimality of the proposed framework. By instantiating cyclic locally recoverable codes (LRCs) as base codes in the framework, the master can recover the desired $K_c$ linear combinations by contacting fewer than $N-S$ workers under many straggler scenarios. This essentially softens the straggler threshold without increasing the computation load or communication cost. Finally, we extend the construction to reduce the per-worker communication load by a factor of $m$, while achieving an asymptotically optimal computation load of $M=\frac{K}{N}(S+mK_c)$.

0.20