CS Paper Radar

Measuring Tabular Data Formats Cold: A Replicated Cold-Cache Benchmark of Serialization, Scan, and Index Performance on Real Datasets

Asra Fathima, Kristoffer Roberts

cs.DB cs.PFcs.SE ·submitted 2026-11-07 02:10 UTC · v2
benchmarkcode/data

Abstract

Choosing a file format for tabular data is a routine engineering decision, yet published comparisons often time reads served from the operating-system page cache, use one session, or rely on a single synthetic dataset. This article benchmarks fifteen format and access-path configurations (CSV, JSON Lines, Pickle, Feather, Parquet, ORC, DuckDB, and SQLite, with several access paths) on two real public datasets, New York City taxi trips and United States airline on-time records, at 50,000 to 3 million rows. Every measurement uses an explicitly cold cache, randomized execution order, and three independent sessions (with restarts on the laptop), and is analyzed with bootstrap intervals and Holm-adjusted permutation tests. Parquet answered a filtered aggregation 2.6 to 11 times faster than CSV, while JSON Lines was 10 to 27 times slower. A secondary index made SQLite queries 2.6 to 4.8 times slower on randomly ordered data and helped only on data sorted by the filtered column, at low selectivity. A row filter inside the Parquet reader was significantly faster than a projected scan on flight data from one million rows and on both datasets at three million, and row-group skipping needs several row groups, which the default writer produced only at 3 million rows. A 4 GB Linux host ran out of memory on configurations that a 24 GB laptop completed and reversed some orderings, such as Pickle versus Parquet. Code, derived data, and raw timings are released.

Comment
Submitted to IEEE Open Journal of the Computer Society

Closest papers

Nearest neighbours by meaning across the whole index.

Transactional Data Change for Real-Time Analytics in Lakehouses

cs.DB cs.AIcs.DCcs.PF ·2026-11-09 ·Dongyang Geng, Haoqiong Bian, Zinuo Li, Haoyue Li
top venuecode/data similarity 0.57
Abstract · Accepted by VLDB 2026 and to be published on PVLDB Vol.19 No.13, 14 pages, 16 figures

Real-time data analytics has become a key demand for lakehouses. It requires propagating data changes from the data source to the lakehouse in real time, while ensuring ACID guarantees and minimizing resource consumption and impact on analytical query performance. This is highly challenging for a data system according to the RUM conjecture, especially given the read-optimized immutable file storage layer in lakehouses. Existing lakehouses offer only git-style catalog version control on a single table, and they trade data freshness, query performance, and/or resource efficiency to achieve a barely acceptable data-change throughput. In this paper, we propose a novel data synchronization framework, Retina, that enables real-time, cost-efficient, query-performant, and fully transactional data changes in lakehouses. We implement and open-source Retina. Evaluations show that Retina delivers 10-millisecond-level data freshness and over 3.2 million row/s linear-scalable data-change throughput with low resource consumption and query-performance overhead, significantly outperforming state-of-the-art lakehouse systems and lake-compatible HTAP databases.

0.49

Revisiting Index Performance under High Cache-Miss Penalties

cs.DB ·2026-11-16 ·Mathew M. Kim, Beomseok Nam
similarity 0.57
Abstract

Modern data-intensive applications often query very large datasets, making index performance critical. Recent index structures exploit the fine-grained access capabilities of emerging storage devices such as DCPMM and CXL. However, existing benchmarks typically evaluate indexes in isolation using saturated, closed-loop workloads that emphasize maximum throughput. Such workloads favor highly concurrent designs while overlooking the application work that follows an index lookup and competes for cache, memory bandwidth, and CPU resources. We present AuxBench - a microbenchmark that evaluates indexes as auxiliary access paths within larger application workloads. Using AuxBench, we evaluate four byte-addressable indexes under controlled post-lookup memory activity, concurrency, and request arrival patterns. Our experiments show that post-lookup memory activity can change the relative performance of competing indexes, that concurrency-oriented designs may lose their advantages when substantial application work is introduced, and that time-varying request arrivals can produce severe tail latency near saturation. These findings demonstrate why index-only benchmarks may not adequately predict performance in larger application workloads

0.22

Same Query, Different Answer: Run-to-Run Variance in Semantic Query Processing

cs.DB cs.AI ·2026-10-09 ·Oktie Hassanzadeh, Dharmashankar Subramanian
code/data similarity 0.57
Abstract · Code and data: https://github.com/semviews/sqp-variance-study

Semantic query engines ask a large language model (LLM) to judge individual rows, and benchmarks compare these engines on the quality of a few executions at temperature 0. We measure how much another execution changes. We re-execute the text and tabular workloads of SemBench, LRO-Bench, and UDA-Bench through their own harnesses, log every one of 1.2 million model calls, and reanalyze SemBench's published executions. At temperature 0, between 0.07% and 10.2% of rows change their operator output over five executions, depending on the model, and up to 24.0% on one task. Plans amplify these changes exponentially in the number of rows an answer exposes, as a possible-worlds model predicts: on UDA-Bench, a few percent of extracted values change, but about two thirds of the answers do. In SemBench's published results, 21% of text results change quality between executions, including on engines whose plans are deterministic. Several per-query system comparisons, and 16 of 28 tests of LRO-Bench's implementation recommendations, are not settled by the executions the benchmarks run. Changing rows are rows on which models disagree with each other; identical requests within one execution disagree as often as across executions; and one execution's decision margin or reasoning length predicts which rows change. Voting reduces the variance but does not remove it, and a calibrated cascade keeps its guarantee while the rows it returns change. We derive reporting rules for benchmarks and design rules for engines.

0.46

Hybrid Product Search Without the Copy: Lexical, Semantic and Filtered Retrieval Served Directly from a Data Lake

cs.LG cs.DBcs.IR ·2026-11-08 ·Raghvendra Yadav, Xiang Fu, Chinmay Soman
benchmark similarity 0.56
Abstract · 6 pages. Submitted to ISIR-eCom 2026 (workshop at IEEE BigData 2026)

E-commerce catalogs increasingly live in lakehouse tables: Apache Iceberg metadata over Parquet files in object storage. Product search, however, still runs on copies of that catalog, in a lexical engine and a vector database, each with its own ingestion pipeline, schema, staleness window and bill. We show that search applications can be hosted directly on the data lake, with the lakehouse table as the single system of record. We extend Apache Pinot, a real-time OLAP engine that already serves Iceberg tables in place, with BM25 relevance scoring over its Lucene text index, corpus-wide term statistics so that scores are comparable across files, and reciprocal rank fusion in plain SQL; a real-time table fed by a stream can sit behind the same table, merged by the broker. On the Amazon ESCI benchmark (1.2M US products, 22,458 test queries), hybrid search served from the lake matches exact retrieval (nDCG@10 .361). The lake table is queryable 149 s after its files are registered, and a lake commit is searchable in 3.8 s, close to indexing directly into a search engine. On a brand-sorted lake of 277M products (0.8 TB), one 16-vCPU node answers brand-scoped vector, BM25 and client-fused hybrid searches in 6.5-9 ms (p50), sustains 1,100 vector and 635 BM25 searches per second at under 180 ms p99, and makes a commit searchable in 4.3 s. Search adds only its indexes: no copy of the catalog, no sync pipeline and no second schema, which cuts storage, rebuild time and the operational work of keeping copies consistent.

0.32

Client-side transparent caching for remote ROOT data analysis

cs.DC hep-ex ·2026-08-31 ·Dmytro Kovalskyi, Jan Eysermans, Mariarosaria D'Alfonso, Christoph Paus
benchmark similarity 0.55
Abstract

High-energy physics analyses often process the same data as physicists refine algorithms and test new ideas. With data increasingly read from remote storage, each iteration is subject to network latency and depends on network bandwidth and shared-storage throughput, which can vary substantially under load. We present uCache (xrd-ucache), a transparent client-side cache implemented as an XRootD client plugin that requires neither server-side deployment nor changes to analysis code. It uses local storage on the analysis machine as a cache layer between the network and memory. The cache stores only the data actually read by an analysis. It can also rebuild cached data into a branch-aligned, recompressed form, eliminating most of the input/output and decompression costs of subsequent passes. We benchmark the cache using the Analysis Grand Challenge top quark pair analysis on public CMS Open Data compressed with zlib and LZMA. Filling the cache adds essentially no overhead compared with a direct read. Subsequent passes are 1.6-8.8 times faster from the byte cache and 2.1-15.7 times faster from the recompressed cache. For a typical analysis, a 1 TB cache suffices for datasets of 10-20 TB. The largest improvements occur when the remote data source is heavily loaded or geographically distant.

0.23

Placing LLM Lookup Tables by Per-Layer Prefetch Deadlines

cs.DC cs.AIcs.LG ·2026-11-10 ·Wen Huang, Jack Liu
similarity 0.54
Abstract · 15 pages, 5 figures, 10 tables

DeepSeek-V4.1's 203 GB of Engram n-gram tables sit in host DRAM today, taking 190 of the 223 GiB a 4-GPU serving process uses. Row indices depend only on input tokens, so reads can start with each step and overlap earlier layers. V4.1's two Engram layers have windows that differ by 30x (0.3 vs. 9.3 ms). We show that this deadline, together with session reuse and contention on a shared device, decides which tier each table can use and which layers need cache, and we give a model that predicts the resulting slowdown from measured read times. Agent and chat traffic reuse rows within sessions, and static hot sets transfer poorly. We build SlackTier, a table tier for SGLang that issues all lookups at step start. It reads the short-window layer first and splits a small GPU row cache across layers. The path runs inside the decode CUDA graph and is bit-exact with the original gather. Layer 14's table can go to NVMe uncached with no measurable decode cost. With both tables on four NVMe disks, mean time per output token on agent and chat replays rises at most 1.4% over unmodified SGLang, and host memory falls from 212 to 34 GiB. Both tables need no cache when read over RDMA (remote direct memory access) from another node's memory; one copy serves two instances. On a KV-constrained agent workload, using the freed memory as host KV cache at the same memory budget raises median throughput by 36% and cuts median p90 time to first token from 38 to 24 s.

0.25

Nemo: Efficient Caching with Index Pushdown for Disaggregated OLTP Databases

cs.DB cs.DC ·2026-10-23 ·Wenjie Hu, Xinjing Zhou, Zhihan Guo, Xiangpeng Hao, Xiangyao Yu
published similarity 0.54
Abstract

Storage-disaggregated databases decouple compute from storage, improving elasticity but suffering from network bottlenecks. Most cloud databases use page-based B+trees and buffer pools. Under disaggregation, each index traversal incurs serial network round-trips. Page-granularity transfers and caching also amplify network traffic and lower cache efficiency. Pushing down the full index to storage and caching only hot records at compute is appealing, but an efficient design must (1) provide a holistic record cache that accelerates both point and range queries, (2) mitigate the negative search penalty of confirming non-existence via remote probes, and (3) enforce phantom protection despite the index being encapsulated by the storage service. We introduce Nemo, a holistic index-and-buffer design optimized for storage-disaggregated OLTP with index pushdown. Nemo employs a tree-structured record buffer that unifies caching and buffering and accelerates both point and range queries. It further reduces the negative search penalty by caching negative search results. Nemo provides phantom protection that requires no knowledge of the storage-side data layout and is robust against large key domains, varying data skew, and partition schemes. Experiments show that Nemo outperforms page-based B+Tree and the state-of-the-art 2-Tree by up to 11.42x and 2.97x on positive point queries, and achieves up to 4.76x speedup on negative searches over a hash-indexed record cache. For phantom protection, Nemo outperforms PCL by up to 1.55x even under a dense key domain. Moreover, Nemo maintains stable performance while PCL degrades sharply as the key domain grows large and sparse.

0.39

Compressing Page-Table Leaves in Shared Caches

cs.AR ·2026-11-18 ·Samarth Narang, Aamer Jaleel, Brandon Reagen
similarity 0.54
Abstract

Hardware page-table walkers fetch descriptors through the cache hierarchy, so page-table walks compete with application data for shared-cache capacity. Retaining leaf page-table lines on chip avoids costly DRAM reads during address translation, yet as a workload's mapped footprint grows, these lines displace application data and slow cache-sensitive co-runners. We introduce LeafPack, a shared-cache organization that resolves this trade-off by compressing architectural page-table leaf lines in place within standard cache ways. By exploiting regularity in the physical frame numbers and attributes of neighboring descriptors, LeafPack packs up to sixteen 64-byte leaf lines into a single 80-byte slot including metadata. It packs leaf lines incrementally as walks fetch them and reconstructs individual lines on demand while leaving irregular lines uncompressed, preserving both the in-memory page-table format and the hardware walk algorithm as long as walkers tag leaf requests. A runtime set-dueling controller adapts page-table occupancy by choosing between compressed and uncompressed way ceilings to balance translation coverage against application-data capacity. We evaluate LeafPack in full-system Arm gem5 simulations across a small and a server-class out-of-order core on workloads spanning one to eight cores. Across nine translation-heavy workloads, LeafPack improves geometric-mean harmonic speedup on the small and server cores by 28.7% and 28.4% over our baseline shared cache, and by 18.7% and 17.7% over an SRRIP baseline. Across twenty unseen held-out mixes evaluated after freezing the controller, LeafPack speeds up every mix with geometric-mean gains of 21.4% and 18.5%.

0.19