CS Paper Radar

Scaling the Lightning Network with Practical Set Reconciliation

Xingyu Chen, Anish Sinha, David Starobinski, Ari Trachtenberg

cs.NI ·submitted 2026-08-16 20:31 UTC · v1
published

Abstract

The Lightning Network (LN) utilizes gossip to share network topology, channel announcements and updates, and node announcements among its local constituents. Yet, our measurements show that this flooding-based gossip reconciliation is fundamentally inefficient. We propose, instead, to use set reconciliation protocols for sharing this information, and we systematically evaluate existing approaches under realistic network conditions. We further propose ADAPTIVEIBLT, a novel adaptive IBLT (Invertible Bloom Lookup Table) protocol with a partial-decoding enhancement. By simulating reconciliation in Core-Lightning and evaluating real gossip snapshots, we demonstrate the practical benefits of reconciliation in scaling gossip reconciliation from hours down to a few minutes.

Comment
Published in the 2026 IEEE International Conference on Blockchain and Cryptocurrency (ICBC 2026)
Journal ref
2026 IEEE International Conference on Blockchain and Cryptocurrency (ICBC), 2026, pp. 1-5
DOI
10.1109/ICBC67748.2026.11575524

Closest papers

Nearest neighbours by meaning across the whole index.

Off-Tree Forwarding for Accelerating Broadcasts from Arbitrary Sources in Plumtree

cs.DC cs.NI ·2026-10-18 ·Ryohei Banno
similarity 0.60
Abstract

Overlay broadcast is fundamental to large-scale distributed systems, enabling messages to be disseminated to all participants. Plumtree is a well-established protocol that embeds a spanning tree in a gossip overlay: payloads are disseminated along tree links, while non-tree links support recovery and optimization. However, because the tree is constructed from the first broadcast, it is optimized for the initial source and can introduce long detours for broadcasts originating at other nodes, particularly leaf nodes. To address this limitation, we propose probabilistic exuding, an off-tree forwarding mechanism in which nodes near the source additionally forward payloads over their non-tree links with a specified probability. Exuded messages use a dedicated message type and do not participate in tree construction or repair, leaving the shared tree unchanged. Simulations on a 1000-node random regular overlay show that, for randomly selected sources and a representative parameter setting, the proposed method reduces the 95th-percentile delivery latency by approximately 40% relative to Plumtree. This closes approximately 92% of the latency gap between Plumtree and flooding while sending only about 58% as many payload messages as flooding. In contrast, Plumtree's own link optimization and shorter repair timeouts, which also exploit non-tree links, rewire the shared tree and increase latency as broadcast sources change; the latency of exuding is nearly unaffected.

0.21

SPICE: Sparse Parameter Gossip for LLM Pretraining

cs.DC ·2026-11-17 ·Pietro Cagnasso, Daniele Kopyshevskiy, Edouard Oyallon, Eugene Belilovsky
similarity 0.56
Abstract · Technical report, 19 pages, 7 figures, 7 tables

Pretraining language models across geographically distributed resources requires communication-efficient methods that operate under limited and variable internet bandwidth. Decentralized training replaces global collectives with peer exchanges, but full-model messages make larger peer neighborhoods costly. We introduce SPICE, which combines sparse parameter gossip with error feedback through corrected parameter values that remain directly interpretable as peer neighborhoods change. These sparse exchanges make broader neighborhoods affordable, achieving better training quality with less communication than the evaluated dense decentralized baselines. As the replica count grows, SPICEincreases the number of peers contacted while keeping individual messages sparse, limiting the quality degradation observed with dense gossip. Under bandwidth constraints, SPICE achieves higher training throughput than DDP and dense gossip, while sustaining throughput during transient link slowdowns.

0.24

IBLTs Measure Before They Decode: Self-Sizing Set Reconciliation for Database Consistency Verification

cs.DB cs.DC ·2026-08-27 ·Min Wu, Ji Qi, Zhengsheng Ye, Chengdui Luo, Shudong Lu +1 more
theorycode/data similarity 0.55
Abstract · Code and archived artifact: https://github.com/whitewum/self-sizing

Cross-system data replication pipelines cannot confirm end-to-end consistency from the local guarantees of each hop, so the two endpoints must be compared directly on a periodic basis. Once the rows of a fixed snapshot are normalized into fingerprints, the task reduces to finding the symmetric difference of the two sets. An Invertible Bloom Lookup Table (IBLT) reconciles the sets with communication that grows only with the difference cardinality $d$, independent of table size, but its capacity must be fixed while $d$ is still unknown. Across 41,603 production reconciliations over 90 days, nonzero $d$ spans about seven orders of magnitude, and no reliable empirical constant exists. We show that the count array of an IBLT has already measured $d$ before decoding. The measurement is in-band: it is carried by the recovery sketch itself and adds no bytes dedicated to estimation. A mapping-aware theorem extends the construction to Irregular, Rateless, and MET IBLTs. The protocol reads the estimate only after a decoding failure; we prove that the failure-conditioned lower quantile bounds the risk of underestimation, which gives the second-round capacity a configurable success-probability guarantee. The resulting self-sizing protocol attempts recovery with a small first-round sketch and stops on success; on failure it reads $d$, sizes the second round, and completes reconciliation in at most two rounds. Against a controlled oracle, communication is 1.29-1.47 times that of a scheme given $d$ in advance. Production workload characterization, relational-database replay, and a cross-city KV deployment confirm the end-to-end mechanism. In production on an Oracle-MySQL link, all completed runs succeeded within two rounds, over 90% on the 1-RTT fast path with a single 16 KB sketch.

0.38

Does Client Diversity Protect the Lightning Network? Evidence from the 2026 Core Lightning Shutdown

cs.CR ·2026-10-14 ·Saurabh Kumar
code/dataunusual similarity 0.53
Abstract · v2: client inference rules v4 (CLN and Eclair separated by feature bit 5 instead of keysend) with a rules table; node shares and reachabilit…

Client diversity is often cited as the Lightning Network's defence against a bug in any one implementation, yet LND runs 80% of public nodes (84% counting one operator's fleet of about 2,000 tiny nodes) and holds 84% of public channel capacity, a capacity share that has barely moved since 2023. We test what diversity buys with a natural experiment. On 26 August 2026, amid security fixes in every implementation, Core Lightning (CLN) maintainers told operators who could not upgrade to disconnect their nodes. Using 805 daily gossip snapshots, 132,360 on-chain channel closes and a client classifier validated against self-identifying aliases and against the wallet that builds each node's funding transactions, we find that 22% of CLN nodes went dark overnight, ten times the usual rate, mostly Tor-reachable ones; a third were still dark five weeks later. The rest of the network barely noticed. Reachability among all node pairs fell by 7.8 percentage points, a loss exceeded in 4 of 94 ordinary two-day windows and half that of a Tor denial-of-service wave in July, while among CLN pairs it fell by 24.5 points, more than in any ordinary window. Calibrated failure scenarios explain the containment: the blast radius depends on whether a failure reaches the hubs, not on how many nodes run the failing software. Losing ACINQ's Eclair node alone costs the surviving network four times the reachability lost with all 619 routable CLN nodes, and minority clients' reliance on LND for connectivity is what their size implies; CLN nodes in fact peer with each other more than chance. Diversity statistics that count nodes mislead; the quantity to track is which software the hubs run.

0.61

On the Robustness of Gossip Protocols in Modern Blockchains

cs.CR ·2026-10-15 ·Wenting Li, Jannik Albrecht, Frederik Armknecht, Ghassan Karame
top venue similarity 0.52
Abstract · Accepted at IEEE Symposium on Security and Privacy (IEEE SP), 2027

Modern platforms such as Ethereum and Polkadot nowadays execute Byzantine fault-tolerant consensus protocols over large validator committees, often comprising hundreds of nodes. At this scale, complete graph topologies for connecting nodes are impractical, leading systems to adopt various gossip mechanisms, often in an ad hoc manner. In this paper, we study the robustness of widely deployed gossip protocols used in committee-based blockchains against realistic network adversaries. We show, both analytically and through extensive empirical measurements, that the graph parameters commonly chosen to instantiate random graphs are insufficient to guarantee that enough honest nodes receive consensus messages for quorum formation in existing protocols. Motivated by this finding, we empirically assess the robustness of Bitcoin's Erdős-Rényi (ER) graphs, Polkadot's grid topology, Ethereum's GossipSub, and Solana's chunk- and tree-based disseminations. Our results reveal surprising tradeoffs: Solana's dynamic trees are bandwidth-efficient in benign settings but fragile under adaptive attacks; Polkadot's grid achieves strong robustness at the cost of high bandwidth overhead; GossipSub is both bandwidth-efficient and resilient to adaptive adversaries, yet highly sensitive to parameter choices; and ER graphs provide a strong robustness-efficiency tradeoff, e.g., when the (average) fanout degree exceeds 15 for networks up to 5000 nodes.

0.29

RW-LoRA: Communication-Efficient Decentralized LoRA Fine-Tuning via Random Walks

cs.LG cs.AI ·2026-08-31 ·Xingran Chen, Rohit Bhagat, Ghadir Ayache, Rawad Bitar, Yanmin Gong +1 more
similarity 0.51
Abstract

Parameter-efficient fine-tuning methods such as LoRA have become a standard approach for adapting large foundation models. Adopting fine-tuning to distributed settings faces several challenges. Most existing distributed LoRA methods rely on centralized aggregation, and gossip-based decentralized LoRA requires repeated synchronization among multiple model copies. Both methods incur significant communication overhead and introduce errors due to simultaneous aggregation of multiple model updates. In this paper, we take a different perspective and propose a random-walk-based LoRA fine-tuning scheme. Instead of maintaining multiple model replicas, a single model token traverses the network and is updated sequentially using local fine-tuning objectives. This design eliminates the need for global synchronization, substantially reduces communication and computation costs, and avoids aggregation errors. We provide rigorous convergence guarantees for non-convex objectives under standard assumptions. Through empirical results on multiple NLP tasks and graph topologies, we show that the proposed method achieves competitive task performance with substantially less communication and computation than gossip-based LoRA.

0.38

Optimal Sampling and Transmission Policy for Information Freshness in Gossip Networks

cs.IT cs.NIeess.SP ·2026-11-07 ·Irtiza Hasan, Ahmed Arafa
theory similarity 0.51
Abstract · Potential IEEE journal publication

We study the optimal sampling and transmission policies for a transmitter communicating with n gossiping receivers with the objective of tracking a source's status over time using a semantic freshness metric: Version Age of Information (VAoI). The transmitter observes the version changes in the source, and can acquire a new information version by paying a sampling cost. The receivers gossip, allowing information exchange among each other without direct transmission. Gossiping can save on transmission cost, but it may propagate stale version information, necessitating transmitter intervention. With imperfect communication links, we formulate an infinite horizon average cost Markov Decision Process (MDP) in order to jointly optimize receivers' VAoIs, sampling and transmission costs in a time slotted fully connected (FC) gossip network. We first solve the MDP using Relative Value Iteration (RVI) algorithm. Then, we prove that the sampling decision has a threshold structure in the transmitter VAoI. Further, under reliable gossiping links, we show that transmission and idling has an VAoI-difference threshold structure with minimum VAoI activation. Numerical results obtained using RVI show that the optimal policy consistently outperforms natural baselines. These results characterize how source sampling, direct transmission, and receiver gossip should be jointly coordinated for cost-aware semantic freshness in gossip networks.

0.13

KCensus: Synthesizing Latency-Optimal Consensus Fast Paths (Extended Version)

cs.DC ·2026-09-25 ·Clément Burgelin, Antoine Murat, Gal Sela, Marcos K. Aguilera, Rachid Guerraoui
published similarity 0.50
Abstract · 38 pages, 15 figures. Extended version of the paper to appear in the 22nd European Conference on Computer Systems (EuroSys '27), Rabat, Moro…

Strongly consistent geo-replication often relies on fast paths to reduce latency in the common case of no failures or contention. Existing fast-path schemes, however, are ad hoc and restrictive: each corresponds to a point in a broad design space shaped by network topology, workload, and latency objective, so no single scheme works best across settings. This paper looks at fast-path schemes from a new perspective, as mechanisms that spread knowledge about proposals. With this view, we identify a fundamental condition on the spread of knowledge for a fast-path scheme to work. We then introduce KCensus, a framework that turns this condition into an optimization problem, synthesizing new fast-path schemes that are optimal for a given setting. We use KCensus to build a geo-replicated key-value store and evaluate it across AWS regions. Our system outperforms competing protocols, with up to 16% lower average latency.

0.56