CS Paper Radar

An Economic Analysis of DNA-based Data Storage Systems

Alex El-Shaikh, Bernhard Seeger, Thomas Heinis

cs.ET cs.ARcs.IT ·submitted 2026-08-26 19:23 UTC · v1

Abstract

Deoxyribonucleic acid (DNA) remains stable for millennia without degradation, can store information at densities orders of magnitude higher than current technologies, and is environmentally friendly due to its low energy requirement. While these advantages make it a promising candidate for storing archival data, DNA storage systems are constrained by high costs, particularly those associated with DNA synthesis. In this paper, we present a comprehensive cost model for calculating the cost of archiving data using various storage systems. Leveraging our model, we conduct a cost analysis of DNA storage versus leading magnetic tape and cloud-based archival storage services. Furthermore, this model can be customised with various parameters to reflect future cost declines and other cost-relevant trends. Our results indicate that, under current cost trajectories, DNA storage costs must fall by eight to nine orders of magnitude to become economically competitive with today's data archival technologies. Moreover, we explore multiple "what-if" scenarios over the coming decades, quantifying the rate of cost decline required to close the competitiveness gap relative to tape and cloud archives. Our findings underscore the critical need for accelerated innovation and investment in DNA synthesis technologies to reduce their cost and transform DNA storage into a practical archival solution.

Closest papers

Nearest neighbours by meaning across the whole index.

The Unified Evaluation App for DNA Data Storage Codecs

cs.ET cs.SE ·2026-08-10 ·Aleksandar Anžel, Chisom Anyabolu, Leon Wimbes, Luca Staus, Ihsan Tri Heldian +10 more
code/data similarity 0.63
Abstract · 38 pages, 12 figures

Background: Deoxyribonucleic acid (DNA) data storage is a paradigm with great potential for ultra-dense and durable information preservation. However, the rapid proliferation of coding schemes, or codecs, each with their own design constraints and reporting practices, has led to a fragmented landscape that lacks a standardized comparative assessment. Methods: We developed an open-source, modular benchmarking platform that systematically integrates and evaluates state-of-the-art DNA storage encoding and decoding methods (codecs). Our approach uses a curated, diverse set of baseline data and applies multidimensional assessment criteria that are aligned with the consensus standard of the DNA Data Storage Alliance. These criteria include encoding/decoding throughput, computational efficiency, error correction performance across substitutions, insertions, and deletions, and cost efficiency. Results: The developed platform integrates standardized wrapper functions for encoding and decoding, allows for the integration of new methods, and automates reproducible evaluations with comprehensive visual and tabular reporting. Benchmarking both contemporary and classical codecs using their default parameters and multiple metrics demonstrates that no single algorithm is optimal across all evaluated dimensions. The trade-offs between information density, success rate, runtime, and cost are quantified and shown to be critical factors in the design of future-proof formats. Conclusions: Our work establishes a rigorously standardized, open-source evaluation framework that enables reproducible benchmarking, supports evidence-based codec selection, and provides the necessary foundation for translating DNA data storage from experimental research into deployable archival systems.

0.26

ACOR: incremental evidence accumulation towards real-time DNA storage reconstruction

cs.IR q-bio.GN ·2026-10-27 ·Linxuan Han, Huaming Wu, Shi Jia, Guanjin Qu
benchmark similarity 0.56
Abstract

DNA offers high-density, durable storage by encoding digital information into synthetic strands and retrieving it through sequencing. Sequence reconstruction combines noisy copies of each strand before downstream decoding. Conventional offline methods typically reconstruct completed read clusters, separating reconstruction from ongoing read collection. Streaming reconstruction offers a way to overlap evidence processing with sequencing and reduce the computation remaining before downstream decoding. We present ACOR, a streaming consensus reconstruction method that records $k$-mer transitions and boundary support in a persistent ledger, then decodes through length-constrained beam search and priority-based selection at cluster closure. Evaluation on real and simulated DNA-storage datasets showed competitive recovery. On our own short-read next-generation sequencing (NGS) wet-lab library, ACOR combined near-complete recovery with second-scale reconstruction of more than ten million reads. A smaller public benchmark was completed in a fraction of a second. Parallel ACOR achieved speedups of tens to over a thousand-fold relative to the tested serial baseline configurations. Sparse storage reduced high-context memory requirements, while parallel execution accelerated reconstruction computation. ACOR is designed for streaming readout pipelines that process assigned reads while sequencing continues.

0.23

HORIZON: A Read-Efficient Firmware for DNA Storage with Horizontal Layout

cs.ET ·2026-08-25 ·Alex Sensintaffar, Roop Kiran, Yang Chen, Mai Zheng, Bingzhe Li
similarity 0.54
Abstract

DNA storage is a promising medium for long-term archiving, but its read performance is limited by coarse-grained random access. Existing random-access DNA storage designs suffer from high read amplification because their sequential layouts co-locate frequently and infrequently accessed data under the same primer pair, where any read must retrieve all associated strands even when only a small fraction is needed. We present HORIZON, a read-efficient allocation policy for DNA block devices that reduces read amplification through activity-aware horizontal placement. HORIZON first introduces a horizontal layout distributing writes round-robin across primer pairs, rather than filling each sequentially. It classifies newly written blocks in the write buffer as active or inactive, tracks recent primer-pair accesses using a sliding-window temperature model, and allocates blocks based on block activity and primer-pair occupancy. Simulations show HORIZON consistently reduces read amplification compared with state-of-the-art schemes across MSR and FIU traces and synthetic filesystem workloads.

0.29

To Store or To Regenerate? A Cost Model for AI-Generated Content at Scale

cs.PF ·2026-09-24 ·Yunjia Zheng, Zirui Wang, Haoran Ni, Tingfeng Lan, Zhaoyuan Su +2 more
similarity 0.54
Abstract

AI-generated content is becoming a rapidly growing class of digital artifacts. Because these artifacts accumulate over time, their exponential growth creates a substantial storage, energy, and infrastructure cost for operators and society. At the same time, GPU compute cost continues to fall rapidly with each hardware generation. This divergence raises a fundamental question: when does on-demand regeneration become cheaper than persistent storage? This paper develops a cost model for comparing persistent storage and on-demand regeneration for AI-generated artifacts. The model accounts for corpus growth, HDD and tape price trends, drive replacement, electricity, request skew, caching, generator FLOPs, and future GPU price-performance improvements. For image generation, our analysis shows that prompt-based regeneration does not become cheaper than storage until around 2040, because every cache miss must still rerun the full prompt-to-artifact generation pipeline. We observe that widely used diffusion-based generation models operate in latent space, creating an alternative point in the cost tradeoff: instead of storing the final artifact or only the prompt, operators can store a compact intermediate representation (IR) and perform cheap on-demand decoding. Our analysis shows that caching combined with IR-based regeneration substantially reduces both storage and compute cost, making it at least 2x cheaper than both full-object storage and prompt-based regeneration even today. On a production image trace with 2.07 billion requests, the same conclusion holds: prompt-based regeneration is over 100x more expensive than storage, while IR-based regeneration reduces total cost to roughly half that of full-object storage while preserving interactive miss latency.

0.20

Storage-Centric System Designs for Enabling Fast, Efficient, and Low-Cost Genomic and Metagenomic Analyses

cs.AR cs.DCq-bio.GNq-bio.QM ·2026-08-31 ·Nika Mansouri Ghiasi
highly cited similarity 0.50
Abstract

Genomic and metagenomic analyses play critical roles in many fields, such as precision medicine, urgent clinical settings, discovering early warnings of communicable diseases, ensuring food safety through pathogen monitoring, agriculture, and scientific discovery. Due to the challenges of analyzing and storing massive volumes of genomic and metagenomic sequence data, significant efforts have been made to accelerate (meta)genomic analyses and store sequence data compressed. Despite the benefits of these techniques, we identify two major outstanding problems in accessing stored sequence data and supplying it to the analysis units: (i) the data movement bottleneck due to moving large amounts of low-reuse data from storage and the unnecessary burden on the rest of the system, and (ii) the data preparation bottleneck, where compressed sequence data needs to be first decompressed and formatted before analysis. In this dissertation, we present customized storage-centric systems, which efficiently (i) analyze (meta)genomic data inside the storage system, and (ii) enable highly-compressed storage and high-performance access of large-scale sequence data, thereby alleviating the overheads of data movement, computation, and data preparation. We demonstrate that the proposed systems significantly improve system performance, energy efficiency, and system cost-efficiency of (meta)genomic analysis. We hope that the storage-centric systems proposed in this dissertation facilitate the broader adoption of (meta)genomic analyses and inspire future research to fundamentally improve the performance, energy efficiency, and cost-effectiveness of other data-intensive application domains related to health and life sciences.

0.51

Transactional Data Change for Real-Time Analytics in Lakehouses

cs.DB cs.AIcs.DCcs.PF ·2026-11-09 ·Dongyang Geng, Haoqiong Bian, Zinuo Li, Haoyue Li
top venuecode/data similarity 0.50
Abstract · Accepted by VLDB 2026 and to be published on PVLDB Vol.19 No.13, 14 pages, 16 figures

Real-time data analytics has become a key demand for lakehouses. It requires propagating data changes from the data source to the lakehouse in real time, while ensuring ACID guarantees and minimizing resource consumption and impact on analytical query performance. This is highly challenging for a data system according to the RUM conjecture, especially given the read-optimized immutable file storage layer in lakehouses. Existing lakehouses offer only git-style catalog version control on a single table, and they trade data freshness, query performance, and/or resource efficiency to achieve a barely acceptable data-change throughput. In this paper, we propose a novel data synchronization framework, Retina, that enables real-time, cost-efficient, query-performant, and fully transactional data changes in lakehouses. We implement and open-source Retina. Evaluations show that Retina delivers 10-millisecond-level data freshness and over 3.2 million row/s linear-scalable data-change throughput with low resource consumption and query-performance overhead, significantly outperforming state-of-the-art lakehouse systems and lake-compatible HTAP databases.

0.49

Creating SquashFS Images to Support HPC and AI Datasets

cs.IR ·2026-10-14 ·Julie Mullen, Albert Reuther, LaToya Anderson, Michael Jones, William Arcand +16 more
similarity 0.50
Abstract · 8 pages, 5 figures, IEEE High Performance Extreme Computing (HPEC) Conference 2026

HPC users often download datasets on HPC systems for analysis and processing. When these datasets are comprised of hundreds of thousands (or more) of small files, accessing them in parallel can put a tremendous burden on the metadata servers of central parallel file systems. If the dataset is read-only, an effective solution is to compress the dataset into a SquashFS image. There are, however, a number of workflows that can be used to create such a dataset image, and we observed that some were more efficient than others. To provide more effective guidance for our userbase, we benchmarked eleven different workflows for creating SquashFS dataset image files of datasets with thousands to millions of very small files. We took into account whether the data arrived as tar.gz, zip, or files-and-directories downloads, whether the image creation was on the central file system or local compute node storage, and the number of SquashFS file compression threads used. This paper shares the results of these benchmarks and provides recommendations to HPC system users toward the best workflows to use for SquashFS image creation.

0.25

USDB: Scaling Storage for Next-Generation Recommender Systems at ByteDance

cs.DB ·2026-11-02 ·Xiaoran Zhu, Jieming Cai, Weiping Yu, Xi Chen, Shu Cai +9 more
similarity 0.49
Abstract · 17 pages, 18 figures, 5 tables

Recommendation models consume increasingly long, attribute-rich user histories, making small continuous updates and repeated large reads a central storage asymmetry. We present USDB, ByteDance's unified sequence storage practice for streaming and batch workloads. CascadeLSM time-based compaction stages materialize application-specific histories; differentiated replicas share persistent files; partial dumps reconstruct inference inputs from captured recent state and versioned history; and a hybrid columnar format accelerates projected reads. These designs reuse maintenance results across ingestion, inference, and training. Production migration records show 85.8% lower mean write latency, while normalized deployment accounting shows 42.1% lower infrastructure cost. Controlled comparisons with an Optimized RocksDB sequence baseline show faster long-history reads across projections and client loads. Aggregation pushdown sustains 8.5x the throughput of ordinary reads under a constrained receive budget. Production reconstruction and direct-file training results demonstrate how shared sequence maintenance supports fresh serving and repeated historical access.

0.19