CS Paper Radar

Reconstructing Missing Mandi Prices in North-East India with Regional Factors

Pritam Deka

cs.CY ·submitted 2026-10-29 12:46 UTC · v1
benchmark

Abstract

Agricultural price panels contain prolonged and clustered reporting gaps, yet their reconstruction is often evaluated by deleting isolated observations. We investigate a regional-factor approach to weekly mandi-price reconstruction in five North-East Indian states, with Bihar and Goa as additional comparison states. The model combines a common price path with persistent market-specific deviations; an adaptive ensemble weights alternative borrowing structures using internally masked reports. On a matched retrospective benchmark against ten baselines, pooled North-East mean absolute percentage error is 19.3%, compared with 26.5% for SAITS. The corresponding calendar-aligned panel/time-block bootstrap interval for the error difference is 2.8-12.7 percentage points. Removing the target state's monthly aggregate leaves pooled error at 19.4%, but eliminates the apparent advantage in Goa. An origin-restricted pipeline combining week-close input processing with revised ensemble validation yields 19.7% error, compared with 30.8% for carry-forward; this contrast does not isolate individual pipeline changes. Gains vary across states, and uncertainty intervals remain too narrow in several small panels. These findings support further study of structured retrospective reconstruction, not deployment claims: model development used earlier results, publication dates for monthly inputs are assumed, and confirmatory evaluation on untouched data remains outstanding.

Closest papers

Nearest neighbours by meaning across the whole index.

Across-Design Uncertainty in Short Pricing Panels: Inference and Identification

cs.LG econ.EM ·2026-08-21 ·Pedro Cadahia Delgado
unusual similarity 0.53
Abstract · 38 pages, 4 figures

Short observational pricing panels often contain many data points but very few actual price changes. This paper shows that this sparsity creates a hidden source of error that standard statistical methods miss. When estimating price effects, most of the uncertainty does not come from sample size within a panel, but from the specific history of price movements observed. Standard confidence intervals fail because they only measure variation within the panel, ignoring this broader design-level error. Using simulations, we find that this cross-design variation accounts for most of the estimation error, causing standard methods to significantly understate uncertainty. First, we show that cross-design error decreases predictably as the total volume of price variation increases. Second, adding more data from regions that share the same price trends does not fix the issue; true precision improves only when combining data across units with independent price trajectories. Third, applying a simple variance-component adjustment across independently priced units restores accurate statistical coverage. We confirm these findings in real-world store scanner data, showing that products and pricing zones behave as if they have far fewer independent price movements than their raw counts suggest. Ultimately, reliable inference in passive pricing data requires genuine, independent variation, which can be achieved through controlled regional price testing.

0.55

Cap-Aware Boosting for Green Day-Ahead Electricity Price Forecasting

cs.LG ·2026-11-16 ·Shubham Pratap
similarity 0.50
Abstract · 19 pages, 7 figures. LightGBM forecasts of 15-minute IEX Green Day-Ahead prices, compared with Chronos-2

Accurate forecasting of Green Day-Ahead Market (GDAM) electricity prices is essential for market participants in the Indian power sector. We present a LightGBM-based forecasting framework for Indian Energy Exchange GDAM prices, which clear in 96 fifteen-minute intervals per day. The proposed system explicitly models price-cap behavior by separating capped-price events from normal market conditions. A classification head estimates the probability of the regulatory price cap (Rs 10,000/MWh), while a regression head predicts uncapped prices in logarithmic space. Final forecasts are obtained through probabilistic blending of these components and are accompanied by nine quantile forecasts for uncertainty estimation. Evaluated on 257 trading days between 1 January and 14 September 2026, the model achieves a mean absolute error (MAE) of 397.9 Rs/MWh and a mean absolute percentage error (MAPE) of 9.57%, substantially outperforming a Chronos-2 baseline (583.3 MAE) and a persistence baseline using the previous day's profile (589.1 MAE). We further introduce leak-safe variants that incorporate real-time market signals, unmatched sell volumes, and solar generation proxies. In a recent out-of-sample evaluation, the quantile-clock variant achieved the best short-term performance with an MAE of 789 Rs/MWh during 25-28 October, while the enhanced capped-price model achieved an MAE of 716 Rs/MWh across 21 evaluation days through 16 November. These results demonstrate the effectiveness of regime-aware probabilistic forecasting for high-volatility electricity markets.

0.24

Price co-movement and price information diverge across Nordic electricity zones after the energy crisis

cs.LG econ.GNstat.AP ·2026-11-12 ·My Thi Diem Phan, Tuyen Trung Truong, Hoai Phuong Ha, Dat Thanh Nguyen
code/data similarity 0.47
Abstract · Supplementary material in the ancillary file. Code and results: https://github.com/myptd/nordic-price-divergence

After the 2021-2022 energy crisis, day-ahead prices in Nordic bidding zones no longer move together. We ask whether this divergence also changes the information carried by prices. The answer matters for integrating variable renewable energy (VRE), because it can help identify where wind and solar forecasts are worth producing, publishing, and using. We measured the co-movement of each Norwegian and Swedish zone with Germany-Luxembourg (DE-LU) in 2018-2025. We then used a single model, selected on 2024 data, to forecast 2025 prices at 11:00 on the previous day and tested which inputs improved accuracy in each zone. The zones moved alike until 2019, but co-movement diverged in Sweden in 2020 and in Norway in 2021, and the split has persisted. In 2024, four southern zones co-moved strongly with DE-LU (intraday correlations of 0.54-0.71), while four northern zones co-moved weakly (0.25-0.30). Yet price information did not follow this split. Previous-day neighbouring prices reduced mean absolute error by at most 2.2% in any zone, equally in coupled and isolated zones. Removing the transmission system operators' wind and solar forecasts increased mean absolute error by at most 6% in Norway, 8-16% in Sweden, and 31% in DE-LU, and the ordering held in 2024. Within Norway, the value of these forecasts showed no detectable dependence on reservoir filling. The divergence in co-movement was therefore not matched by a divergence in price information, which differed mainly between national generation systems. Thus, a zone's coupling to the continent does not indicate the value of VRE forecasts there.

0.40

Crop Yield Prediction for Punjab, Pakistan: A Tree-Ensemble and Leaf-Health Prototype, and What Random Validation Hides

cs.CV ·2026-10-05 ·Amina Asif, Qurat Ul ain Asif, Noor Bakhat Asif, Kasa-e-Fatima
similarity 0.46
Abstract

Yield forecasts help planners and farmers decide on inputs, storage and imports, but small agricultural tables can make reported accuracy fail on a new season. We built a crop-yield prototype for Punjab, Pakistan that combines Random Forest, XGBoost, support vector regression and a Ridge-stacked ensemble with a MobileNetV2 leaf-health classifier, and deployed it as a web application with SHAP explanations. A random 80/20 split of a merged Kaggle-derived table (414 rows) gives the ensemble an R2 of 0.991. An audit showed that the table contains only 46 independent observations: a join with nine temperature records per year copied every crop-year nine times. Holding out whole years takes XGBoost on the same rows from R2 = 0.994 to -0.20. On deduplicated data, and on a longer FAOSTAT table (1990-2024, 70 observations), a per-crop linear trend (leave-one-year-out RMSE 0.29 Ton/Ha) beats every model not given the year (0.84-1.06). Neither pesticide use nor national temperature change explains the trend residuals. An apparent pesticide gain in the Kaggle table disappears on FAOSTAT, where the pesticide series is mostly imputed and the two releases disagree. An independent district-level wheat panel (36 districts, 13 seasons) shows that most variation in Punjab is spatial and that a district mean with a common trend matches the learned models. The leaf classifier reaches 99.87% accuracy on held-out PlantVillage images and recalls 96.4% of 336 unseen diseased leaves after near-duplicates were removed. However, no leaf image is paired with a yield record, so the health score used in the yield models had to be constructed and adds nothing. We report these negative findings together with the prototype.

0.21

TRACE: A Reproducible Benchmark for Electricity Price Forecasting with Official Operational Text

cs.LG stat.ML ·2026-09-30 ·Xinyi Yi, Zhangdie Yuan, Ioannis Lestas
publishedbenchmark similarity 0.45
Abstract · Accepted to the NeurIPS 2026 Workshop on Foundation Models for Time Series (FMTS)

Electricity price forecasting (EPF) supports scheduling, bidding, and risk management in electricity markets, yet existing benchmarks focus mainly on numerical inputs, leaving the forecasting value of forecast-time textual context insufficiently evaluated. We introduce TRACE, a reproducible benchmark of 7,300 zone--day instances pairing prices from five zones in a major U.S. market with official operational text available at the forecast cutoff. TRACE reconstructs official operational text at each cutoff, preventing post-cutoff information leakage. We evaluate TRACE for semantic alignment and forecasting value. Semantic assessments align with central movement and both tail risks in ground-truth prices, most consistently for upper-tail price risk. Forecasting value is reflected in a median 7.4\% reduction in upper-tail pinball loss across time-series foundation models. A controlled cross-day text-mismatch ablation reverses the gains, falling below the no-text baseline.

0.47

When AI Advisors Agree: Distinguishing Forecast Risk from Forecast Reliability

cs.MA ·2026-10-09 ·Russlan Ramdowar
code/data similarity 0.44
Abstract · Version 2: 24 pages, 11 figures, 12 tables. Expanded aggregation, dependence and reproducibility diagnostics; unchanged outcome cutoff and c…

If seven AI advisors agree, what have we learned: that their forecast is reliable, or that the asset is easier to forecast? We examine 928 completed first-quarter outcomes for 232 assets across four historical iPulse AI batches. Ten Gemini configurations form seven equally weighted investment frameworks; no forecasts are regenerated. The council has mean absolute return error of 15.15 percentage points (pp), versus 14.99 for no change, with an unresolved paired difference of +0.16 pp. Error rises from 10.31 to 21.05 pp across disagreement quartiles, but the no-change baseline rises almost identically, from 9.89 to 21.15. In the pooled archive, selecting the lowest-volatility quarter produces lower error than selecting the least-disagreeing quarter: 7.97 versus 10.31 pp. Directional agreement introduces a second trap: its apparent value depends on the aggregation rule. When at least six of seven frameworks share a direction, the sign of their mean return is correct in 53.21% of cases, versus 44.11% with lower agreement. Scoring the modal direction instead changes the lower-agreement accuracy to 54.27%, removing that positive contrast. Recomputed-rank bootstraps, scale-aware losses, and period sensitivities qualify these descriptive results; four overlapping periods cannot establish future-regime performance. Agreement, visible asset risk, and aggregation-specific skill should be evaluated separately before consensus is presented as confidence.

0.61

PaMIR: Open Benchmark of Public Credit-Default Datasets

cs.LG ·2026-10-02 ·Mikhail Liashkov, Ilyas Varshavskiy, Shuhratjon Khalilbekov, Azizjon Azimi, Bonu Boboeva
benchmark similarity 0.44
Abstract

We release PaMIR (Public Arrival-ordered Measurement for Inference in Risk), an open benchmark for credit-default prediction when labels are scarce and arrive late. The field's reference benchmark studies use eight datasets each, only two or four of them public. PaMIR brings together 19 public datasets with binary default labels -- 1.24M loans, firms and card accounts from nine countries -- rebuilt from pinned source snapshots by one leakage-audited recipe and never redistributed; to our knowledge it is the one of its kind as of today. Every model is a single function, scored under a repeated i.i.d. split and a label-delayed stream in which each application is scored on arrival, with AUC reported by label budget; fleet means are withheld unless every dataset is scored. A synthetic-data harness tests generated training rows without letting a generator see held-out rows. This report describes release 0.4.0 of this living benchmark.

0.23

Robustness Evaluation of GPU Cloud Pricing Benchmarks under Incomplete Observations: A Controlled Missingness Study

cs.AI ·2026-11-13 ·Jooya Wong
benchmark similarity 0.44
Abstract · 5 pages, 3 figures, 7 tables

Daily GPU rental-price benchmarks can fail when marketplace quotes are missing. This article stress-tests a fixed equal-weight median rule under four imposed masking mechanisms and four benchmark-level recovery methods. Across 5,800 Monte Carlo realisations in 27 scenarios on 190 held-out days, whole-source outages reduce publication coverage to 0.77 while preserving zero conditional error on published dates; composition-changing loss retains 0.86 coverage but produces 7.74 log points of error. A separate past-only stratum-level ablation reduces error by 4.15 to 6.70 log points in five composition-changing cases, using a richer information set. An observable-signal withholding rule improves the primary error-coverage criterion in 0 of 10 cases. Results concern observed asking-price measurements, not transaction prices or identified real-world missingness mechanisms.

0.37