CS Paper Radar

Interpreting Timestamp Permutations in Dynamic Link Prediction

Ilia Karpov, Kirill Koloskov, Valeriy Kuleshov, Dmitry Ilvovsky

cs.LG cs.AIcs.LOcs.SI ·submitted 2026-11-18 03:08 UTC · v1
code/data

Abstract

Timestamp permutations are used to test whether dynamic link predictors exploit temporal information, but a performance gap can reflect changes in training, evaluation history, or both. We separate these effects with a crossed diagnostic and measure input exposure: whether altered histories actually change the predictor's inputs. On MOOC, TCL trained with binary cross-entropy gains approximately eight pairwise-accuracy points from original-chronology training under either evaluation history, while changing history at fixed weights gives only 0.03 points. The small history contrast reflects unchanged inputs for 68.37% of queries, stable rankings for most others, and cancellation of gains and losses. Under original evaluation history, suppressing explicit time encoding reduces the training contrast from 8.03 to 3.64 points, a paired reduction of 4.39 points (95% interval [3.68, 5.09]) across six seeds. DyGFormer shows corresponding training and history contrasts of 22.34 and 0.38 points across two seeds, despite greater exposure. Partial history permutations further separate nominal intervention strength, input changes, and aggregate accuracy. These comparisons turn a single permutation gap into interpretable statements about training conditions and model responses, while leaving the internal mechanisms of temporal learning open.

Comment
5 pages total (ACM sigconf format). Companion code and trained models: https://github.com/karpovilia/tgn

Closest papers

Nearest neighbours by meaning across the whole index.

What Does History Add? Calendar Controls and Information-Timing Audits for University Trajectory Forecasting

cs.LG ·2026-10-15 ·Zepei Cao
similarity 0.63
Abstract · 12 pages, 3 figures, 6 tables. Methodological simulator evaluation with information-timing and calendar-control audits

We audit the predictive value of historical summaries in a rule-based university trajectory simulator. Seventeen ridge specifications and persistence separate current plans, past-state and past-plan summaries, and calendar-only history-length indicators. Across training, validation and two family-held-out evaluations, we reconstruct 16,750 routes and 804,000 monthly records. On the original 40-family set, six-month history lowers six-step normalized RMSE from 0.159234 to 0.151522, but its difference from a length-only control is -0.000895 (descriptive paired 95% interval [-0.001957,0.000248]). Fixed weights on 30 families unused in identified prior tasks reproduce this pattern: 0.156480 to 0.150074 versus current-state prediction, but -0.000558 [-0.001783,0.000728] versus the length control. These uncertain comparisons do not generalize to all-history summaries, which improve on their own calendar control. Exploratory decomposition finds clearer preparation-index and funds benefits, whereas fatigue dominates the six-month model's loss and has an uncertain length-controlled benefit. Moreover, 28.20% of its six-step fatigue predictions violate the simulator's [5,90] support. We document family dependence, prior exposure, strict temporal indexing, matched recursive references and independent numerical replay. The task conditions on recorded future plans; it does not evaluate real-student forecasts, causal action effects, autonomous planning or language-model memory. This is an evaluation audit, not a new memory architecture.

0.25

Learning the Past from the Future: Retrospective Reconstruction in Temporal Interaction Networks

cs.SI cs.LG ·2026-10-23 ·Amarnath R
similarity 0.61
Abstract · 12 pages, 5 figures, 3 tables

Temporal link prediction uses past interactions to predict future links. In practice, interaction records from web and mobile applications can contain gaps caused by logging failures, API limits or lost records, even when later events remain available. To our knowledge, the extent to which later events help recover interactions within such gaps has not been measured directly for this ranking task. We study retrospective reconstruction using the tgbl-wiki and tgbl-comment datasets. We hide a two-hour window, use its interactions as ground truth, and rank the directed pairs active within it. We compare count-based heuristics and logistic regression models using past information alone or together with 1, 6 or 24 hours of later events. Target pairs and candidate lists remain fixed across conditions, so differences in ranking performance cannot be attributed to changes in candidate selection. With 24 hours of later events, the 26-feature logistic regression improves mean reciprocal rank (MRR) by 0.039 on tgbl-wiki under exhaustive ranking, 0.046 under historical-random ranking, and 0.108 on tgbl-comment under historical-random ranking. The 26-feature logistic regression benefits from adding later features to past features, whereas replacing the past pair count with the later pair count reduces MRR by 0.157 on tgbl-wiki under exhaustive ranking. A control removing later interactions from query sources to historically observed destinations reduces the MRR gain by 59% on tgbl-wiki and 10% on tgbl-comment. Because this control removes more than repeated target pairs, these reductions cannot be attributed to pair recurrence alone. Later events improve the ranking of hidden interactions on both datasets, including tgbl-comment, where only 6.4% of target pairs reappeared within 24 hours.

0.18

Your Temporal Link Predictor Is Blind to Who Is Active: A Missing Factor That Transfers Across Models

cs.LG cs.SI ·2026-10-04 ·Ji Zhang, Zixin Liu, Yiran Ding, Jiayi Wang, Yilu Du +1 more
theorycode/data similarity 0.60
Abstract · 62 pages, 6 figures, 19 tables

An interaction has two parts: someone decides to act, and then chooses whom to act on. Temporal link prediction has concentrated on the second, and we show that it is blind to the first by construction: a standard negative keeps the real source and swaps the destination, and we prove that this cancels the source's activity exactly from the optimal score, so no model trained and evaluated this way is ever rewarded for learning it. Under the harder historical and inductive negatives, whose sources differ, the same factor becomes the dominant signal. We model it with Source Node Activity Modeling (SNAM), a self-exciting event intensity fitted by an exact point-process likelihood to decayed interaction counts the history states already contain; it has fewer than 20 parameters. On their own, never looking at the destination, these parameters beat DyGFormer and TPNet on four of five datasets under historical negatives. Added to the frozen scores of TPNet, TGN, DyGFormer and DSRD, four models of different design, without retraining anything, they raise AP on almost every backbone-dataset pair in both settings, by up to 25 points. Our full model ranks first overall against eleven baselines on 13 datasets and three protocols, and on million-event streams trains an epoch 9-100x faster than TPNet and DyGFormer. We conclude that source activity is a blind spot of temporal link prediction, and a cheap, transferable one to close. Code is available at https://github.com/Erutaner/Your-Temporal-Link-Predictor-Is-Blind-to-Who-Is-Active.

0.64

A Later Test Set Is Not a New Domain: Pretraining Familiarity Survives a Contamination-Free Hold-Out

cs.LG ·2026-09-09 ·Mahdi Naser Moghadasi, Faezeh Ghaderi
code/data similarity 0.59
Abstract · 11 pages, 2 figures, 5 tables. Code, data fetchers and per-series results: https://github.com/mahdinaser/tsfm-bench

Time-series foundation models are evaluated almost exclusively on public archives that predate them, so a strong score cannot be separated from having seen the test set during pretraining. The obvious remedy is a hold-out that postdates the models. We build one: thirteen forecasters -- four classical, three trained per dataset, six pretrained -- on seven groups drawn from five domains, every observation published after the last model was released, and every dataset rebuildable without an API key. Under this protocol pretrained models win 5 of 7 groups, lose one to a Theta baseline, and on daily exchange rates are indistinguishable from a seasonal naive forecast, along with every other method tested. We then ask what separates the wins from the losses, and report a negative result: the two intrinsic properties one would reach for -- seasonal strength and spectral entropy, measured on the input window -- do not account for the pattern, and seasonal strength is if anything negatively associated with the advantage. What does track it is corpus familiarity. Our largest gain (28% lower MASE than the best classical method, on weekly Wikipedia pageviews) falls on Wikipedia pageviews, the domain TimesFM's authors describe as the bulk of its pretraining corpus, at the same granularities and differing only in time window. Within the pretrained family, where every model forecasts identical series so that series difficulty cancels, the TimesFM family outranks the Chronos family by -0.53 ranks on Wikipedia against -0.09 everywhere else (1,500 vs. 754 series, Mann-Whitney p < 1e-5). We conclude that a temporal hold-out removes memorisation of a window but not familiarity with a domain, that benchmarks therefore need domain hold-outs stated relative to disclosed corpora, and that the practitioner's question is less which model is better than whether their domain is one the model was raised on.

0.35

Time-Series Foundation Models That Understand Data Revisions

cs.LG cs.SE ·2026-09-23 ·Taimoor Ahmad
similarity 0.59
Abstract

Historical observations are not always fixed: statistical agencies revise previously published values as new evidence arrives. Forecasting from a contemporary download can therefore expose a model to information unavailable at the date it purportedly made a prediction. We propose VINTAGE-TS, a revision-aware adaptation of a time-series foundation model that distinguishes observation time from information-availability time. Its targets are the next period's first-published value and the value available a fixed number of days after that publication; neither is declared final truth. A joint predictive distribution preserves dependence between these targets and exposes uncertainty about their difference. We specify an ALFRED-based rolling evaluation, a matched Chronos-2 comparison, conventional and revision-aware baselines, and a separate audit of pretraining overlap. The accompanying software implements validity-interval reconstruction, delayed-label filtering, a frozen-backbone adapter interface, and reproducible diagnostics. An executed synthetic demonstration and a 25-configuration sensitivity suite verify the workflow, expose variation across seeds and revision regimes, and illustrate how hindsight contamination changes measured performance. Thirty one automated tests check temporal and integration contracts. Real ALFRED and Chronos-2 experiments have not been executed; no empirical foundation-model advantage is claimed.

0.28

Beyond Update Counts: A Control Map for Evaluating Test-Time Model Updates

cs.LG cs.CL ·2026-10-26 ·Zehao Zhao, Shangjian Yin, Rui Liu, Chongyang Bai, Mark Ibrahim +4 more
theory similarity 0.58
Abstract · 44 pages, 6 figures, including appendices

Test-time updating lets a language model change persistent state after training---such as model weights, adapter parameters, or a memory module---before or while answering queries. Existing evaluations often compare systems using update count or frequency and performance measured immediately afterward. These summaries omit what is updated, where, when, and in what order. The same total can therefore hide an important difference: one system may answer after one update while another answers after two. We introduce the Control Map, an evaluation framework that records the committed updates preceding each scored answer and separately evaluates behavioral changes, immediate learning (acquisition), later recall (retention), preservation of unrelated behavior (locality), and performance in new settings. We prove when counts alone determine final state and show, under stated assumptions, how differences between pre-answer histories limit differences in average scores. Using ControlMap, our experiments reveal three insights. First, update scheduling matters: at a 760M-parameter checkpoint, 24 schedules with the same update count yield 19.7%--34.2% accuracy. Second, learning an edit does not guarantee that it lasts or remains contained: all six selected CounterFact LoRA settings learn the new information, while ROME retains edits at all 13 operating points but fails locality at each. Third, no editor performs best consistently across models, benchmarks, and tests. Evaluations should therefore report update histories and assess acquisition, retention, and locality separately.

0.19

When does a network's training history predict its future learning better than its current state? Evidence from a response probe and a forecasting screen

cs.LG ·2026-10-07 ·Martin Hofmann, Patrick Mäder
similarity 0.57
Abstract · 18 pages, 6 figures, 5 tables

Networks that behave alike now can still learn differently when training continues. Work on loss of plasticity and critical periods shows that the path to a state shapes what follows; it does not show whether the path carries information that a measurement of the state itself misses. We ask when the training history of a network predicts its future learning better than its current state. In a main study, small multilayer perceptrons were trained under three history regimes (42 histories), and future learning was measured at four checkpoints by a short probe: a copy of the network trained for 100 updates on a new task. Before the prediction result was read, the protocol checked the probe. It responded monotonically to a function-preserving rescaling of hidden units, repeated measurements agreed (intraclass correlation 0.940, [0.903, 0.997], in the least reliable class, mean of three repeats), and a re-initialisation of units was visible directly after it but not 100 to 200 updates later. A history state of at most four dimensions did not improve on a calibrated model of the current state (gain -21.4%, 90% interval [-91.9, 8.1]; required in advance: 10%). A companion screen on 1,560 synthetic regression runs asked the same question for a target further away, the final error of the run. There, history models forecast better than the current validation error after 12 of up to 240 epochs (compact state 30.3%, [15.8, 39.4], a contextual comparison) and were not distinguishable from it after 48. In both studies the history was informative only while the current state was not yet informative about the target; this reading was formed after the results.

0.27

TimeLitmus: A Diagnostic Benchmark for Cross-Modal Understanding and Explanation Faithfulness in Event-Conditioned Time-Series Prediction

cs.AI ·2026-09-21 ·Jie Gong, Maowei Jiang, Zhiwei Liu, Yankai Chen, Guojun Xiong +4 more
benchmark similarity 0.57
Abstract

Large language models (LLMs) are increasingly used to make predictions from numerical time-series histories and textual events. Yet accuracy alone cannot reveal whether correct answers reflect effective integration of the two inputs or instead arise from event polarity, unimodal priors, or superficial cues. Likewise, plausible explanations may rationalize predictions without faithfully reflecting the evidence that drives model behavior. We introduce TimeLitmus, a diagnostic benchmark for cross-modal understanding and explanation faithfulness in event-conditioned time-series prediction. TimeLitmus contains 4,856 evaluation records across Finance and Traffic, combining natural prediction with controlled counterfactual and contrastive interventions, explanation-targeted faithfulness tests, and systematic shortcut controls. Across ten representative LLMs, standard prediction accuracy substantially overstates reliable cross-modal understanding: Hard Paired Contrast (HPC) pair correctness peaks at only 19.2% in Finance and 11.7% in Traffic, and all ten models show lower-than-expected consistency on Finance series-side controls. Models often recognize scenario relations explicitly yet fail to apply them during independent prediction. Explanation faithfulness shows a similar gap: in Traffic, most models cite the manipulated temporal factor in over 90% of cases, while behavioral support remains below 22%. Human annotators outperform LLMs on matched controlled and hard-pair diagnostics, confirming that these distinctions are recoverable from the inputs. Natural-only adaptation yields selective gains in evidence selection and input sensitivity, but not consistent gains in controlled or hard-pair behavior. The benchmark, evaluation suite, and supervised adaptation data will be released publicly.

0.27