CS Paper Radar

Green BOA: Determining the environmental break-even point for ML-based data compression

Caterina Doglioni, Thomas Elliott, Akshat Gupta, Hanzila Hussain, Sanjiban Sengupta, Zhengkai Sun

cs.LG hep-exphysics.comp-ph ·submitted 2026-08-20 13:08 UTC · v2
unusual

Abstract

We summarise the outcome of two summer internship projects based at the University of Manchester, focused on the break-even point in terms of environmental sustainability for ML-based data compression algorithms. Using the example of a ML-based lossless compression algorithm, we compare estimates for the carbon-equivalent of the infrastructure needed for ML training and inference with the carbon-equivalent savings from reduced disk storage requirements, and discuss their break-even point.

Comment
3 pages, 1 figure. Accepted as a lightning-talk contribution at the 2nd International Workshop on Low Carbon Computing (LOCO 2026), Lancaster University, United Kingdom, 10-11 September 2026. Part of the LOCO 2026 proceedings, arXiv:LOCO2026/L05

Closest papers

Nearest neighbours by meaning across the whole index.

PointZip: Statistically Lossless Compression for Climate Data with Pointwise Error Bounds

cs.CE ·2026-11-16 ·Jiayong Li, Pratyai Mazumder, Yuejiang Yu, Langwen Huang, Alexandru Calotoiu +1 more
similarity 0.58
Abstract · 12 pages, 11 figures

High-resolution climate simulations generate data volumes that increasingly strain storage, I/O, and data dissemination. Error-bounded lossy compressors can reduce these costs, but their tolerances are usually user-chosen and difficult to standardize across heterogeneous climate datasets. We address this limitation with statistically lossless compression, where the pointwise compression error is bounded by the pointwise data uncertainty. We propose PointZip, a two-stage compressor that combines a rate-distortion-efficient base layer with error-bound-aware residual coding to enforce pointwise guarantees. On outputs from the widely used climate modeling framework ICON, PointZip makes 1km simulations practical by reducing storage bandwidth requirements by 72x. On the ERA5 climate dataset, PointZip reaches 69.3x on the most representative variables, reducing the commonly used subset to 4 TB. In downstream evaluations, PointZip preserves data spectra, extreme events, and air parcel trajectories, while increasing neural weather forecast RMSE by less than 1%, with negligible impact on the Fractions Skill Score (FSS).

0.30

Beyond State-of-the-Art: Standardising Environmental Impact Metrics for AI Research

cs.AI ·2026-10-01 ·Lachlan McGinness, Dan Pagendam, Robert Offner
similarity 0.56
Abstract

As the capabilities and ubiquity of Large Language Models (LLMs) grow, so does their environmental footprint. Despite calls for responsible AI, the machine learning community lacks standardised practices for carbon accounting. Our automated literature review of the 5,285 papers accepted to NeurIPS 2025 reveals that reporting of environmental impact is nearly non-existent. To catalyse a shift toward sustainable AI, we define standardised sustainability metrics for evaluating model training efficiency, accompanied by simple heuristics to estimate the carbon cost of LLM inference. We implement these metrics in carbonbenchmark, a drop-in software solution for tracking and reporting emissions. Finally, to combat the pursuit of marginal accuracy gains at disproportionate environmental costs, we formalise the `Smallest Model that Achieves the Job' (SMAJ), a framework which challenges the field to prioritise computational efficiency and environmental accountability alongside traditional `State-of-the-Art' (SotA) accuracy.

0.04

Improving Progressive Compression with Adaptive Interpolation and Coefficient Decomposition

cs.DC ·2026-09-04 ·Wenbo Li, Xuan Wu, Qian Gong, Pu Jiao, Jieyang Chen +4 more
similarity 0.56
Abstract

Exascale simulations generate data far faster than it can be stored or analyzed, making efficient data reduction essential. Error-controlled lossy compression offers high compression ratios under user-specified error bounds, but the target tolerance must be fixed at compression time. Progressive compression relaxes this restriction, yet existing methods still rely on fixed refactoring strategies and do not fully exploit correlations among decomposed coefficients, limiting the efficiency of progressive retrieval. In this work, we present an adaptive progressive compression framework that improves retrieval efficiency for two common targets, namely error-bound and peak Signal-to-Noise ratios. Our contributions are fourfold. (1) We propose to leverage two complementary interpolation schemes for adaptive progressive compression toward different targets, and we optimize them to achieve high efficiency. (2) We propose coefficient decomposition, a novel method that exploits the commonly overlooked spatial correlations among decorrelated data, which further improves the efficiency. (3) We develop the adaptive progressive compression workflow with automatic selection of the best-fit refactoring pipeline and tailored optimizations. (4) We evaluate the proposed framework on five real-world scientific datasets against three state-of-the-art progressive compressors. Experimental results demonstrate that the proposed framework improves the compression ratio by up to $42.3\%$ under the same requested error tolerance and up to $92.5\%$ at the same PSNR, compared with the best-performing existing methods. When transferring $512$ GB of scientific data to remote sites, the framework delivers up to $1.26\times$ speedup in the end-to-end data transfer performance. Furthermore, our method achieves the highest visualization quality while retrieving the least amount of data from storage.

0.12

A Checklist to assess the energy and carbon impacts of ML/AI applications in Earth System Modeling

cs.AI physics.ao-phcs.LG ·2026-09-01 ·Filippo Dainelli, Amirpasha Mozaffari, Marina Castaño, Aina Gaya i Àvila, Lluís Palma Garcia +3 more
similarity 0.53
Abstract · 12 pages, 1 figure, 2 tables; Submitted and presented at the GREEN-AI workshop of the ECML PKDD 2026 conference in Naples

As machine learning and artificial intelligence find their way into nearly every aspect of climate, weather, and Earth system modeling, it is worth pausing to consider what our design decisions imply for the science and for the computational resources we consume. A growing body of literature addresses the ethical and sustainable development of ML/AI, yet translating these principles into day-to-day research practice remains a challenge as most of best practices are dispersed across multiple studies and commentaries. Here, we distill these discussions into a practical checklist that ML/AI and Earth system science practitioners can use to assess and reduce the environmental footprint of their own applications, organised around the successive stages of the model development pipeline. We complement the checklist with a selection of metrics drawn from the literature for estimating the energy consumption and carbon footprint of a project. For each question, we point to concrete examples and actionable suggestions from recent literature, aiming to bridge the gap between aspirational principles and the decisions researchers face at every stage of the development cycle.

0.37

The Case for Synthesized Data Compression

cs.DB ·2026-10-07 ·Ivan Iachnyk, Mihail Stoian, Luisa Neubauer, Ryan Marcus, Andreas Kipf
similarity 0.53
Abstract

Compression is a fundamental tool in data management. Today's compression schemes are generally designed by hand to apply to many different types of data (e.g., run length encoding or arithmetic encoding), leading to a zoo of general-purpose algorithms which work well for many datasets, but may not be optimal for any specific dataset. Inspired by recent advancements in agentic coding, we propose synthesizing custom-tailored data compression algorithms on a per-dataset basis. Unlike prior work using agents to synthesize database components, a data-specific compression algorithm can be fully verified at construction time, alleviating most correctness concerns. Modern file formats, like AnyBlox and F3, allow including the synthesized code alongside the data itself. Experimentally, we show that our synthesis agent can expand the compression-factor/decompression-speed Pareto front, outperforming general-purpose methods by nearly $2\times$ in compression factor.

0.50

An Introduction to Compression-Based Machine Learning

cs.LG ·2026-09-18 ·John Hurwitz, Edward Raff, Charles K. Nicholas
publishedsurvey similarity 0.52
Abstract · To appear in The 13th IEEE International Conference on Data Science and Advanced Analytics (DSAA 2026)

Any lossless compression algorithm (like gzip) may be converted into a machine learning method, via either Normalized Compression Distance or the Minimum Description Length principle. Any auto-regressive model may be converted into a lossless compression method via entropy coding. This seemingly circular dependence has unrealized potential in modern artificial intelligence and machine learning, and we survey and formalize the various strategies that have been used to leverage compression for machine learning. We introduce and empirically validate a design framework for compression-based ML, finding compression-based methods competitive with conventional baselines and decisively stronger on malware. We find that varying these design choices yields accuracy gains of up to 0.62.

0.37

LOCAA: An Agentic System for Automated Lossy Compressor Tuning

cs.DC ·2026-10-07 ·Khondoker Mirazul Mumenin, Dong Dai, Sheng Di, Franck Cappello, Jinzhen Wang
similarity 0.52
Abstract

Large-scale scientific simulations generate substantial data volumes, making lossy compression essential for reducing storage and data movement costs. However, users configure compressors through numerical error bounds (EBs) while often evaluating results using quality metrics and end-to-end performance. Because the relationship between an EB and these outcomes varies across datasets and compressors, identifying a suitable configuration typically requires exhaustive search, which can be time-consuming and computationally demanding. We present LOCAA, an Large language model-based scientific lossy compression auto-tuning agent that performs compression-in-the-loop search using tool-integrated execution, compressor-aware guidance, and persistent memory. LOCAA supports user-defined objectives and constraints without requiring a specialized search strategy. We evaluate LOCAA across three scientific applications, five compressors, and three use cases: fixed-ratio compression, compression tuning under multiple quality constraints, and compression tuning under quality and time constraints. For fixed-ratio search across twelve fields from two applications, LOCAA requires 1.98x fewer evaluation trials than binary search and 5.03x fewer than FRaZ on average. For compression ratio maximization under joint Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure constraints across six fields from the Community Earth System Model application, LOCAA reduces the average evaluation trials from 61.5 using binary search to 17, corresponding to a 72.4% reduction. Persistent memory further reduces the average number of trials by 27.9% across multiple timesteps of the same field. These results demonstrate the potential of tool-augmented LLM agents to provide flexible and efficient compressor tuning across diverse scientific data and user-defined objectives.

0.22

Towards Sustainable Federated Learning via Carbon-Aware Client Selection

cs.DC ·2026-11-06 ·Wentao Gao, Omid Tavallaie, Albert Y. Zomaya
similarity 0.52
Abstract

The Information/Communication Technology (ICT) industry is severely challenging the global environment due to the sharply rising carbon emissions generated by machine learning (ML) workloads. Federated Learning (FL) is one of the prominent distributed ML systems used in applications with billions of client devices, such as Google Gboard and Apple Siri. Nearly half of the total carbon footprint in FL originates from client-side computation during local training. However, existing client selection strategies in FL primarily focus on minimizing energy consumption rather than carbon emissions, despite the fact that carbon impact depends not only on computational load but also on the underlying carbon intensity of the energy source. In this paper, we address this gap by proposing a carbon-aware fair client selection strategy (CA-FCS) designed to minimize the carbon emissions associated with model training on client devices. In contrast to existing approaches for reducing carbon emissions in FL, we incorporate hourly carbon intensity and a realistic energy consumption model developed using empirical measurements from Google Pixel smartphones to estimate client-side carbon emissions in FL. Since repeatedly selecting low-carbon clients may reduce data diversity and degrade global model performance, particularly under non-independent and identically distributed (non-IID) data settings, CA-FCS incorporates fairness into client selection to mitigate this issue. We conduct extensive experiments under realistic FL settings for mobile devices, where client participation is constrained by device availability conditions, such as charging status and Wi-Fi connectivity. The results demonstrate that our method reduces client-side carbon emissions by up to 46% while maintaining competitive model accuracy.

0.12