CS Paper Radar

ChronicleRec: Pre-training Temporally Anchored Tokens for Lifelong User Modeling

Chengkai Huang, Yubin Sheng, Liang Guo, Haoxi Liu, Junwei Pan, Shangyu Zhang, Zhixiang Feng, Chao Zhou, Chengguo Yin, Lina Yao, Haijie Gu, Jie Jiang

cs.IR ·submitted 2026-09-11 02:47 UTC · v1

Abstract

Modeling ultra-long user behavior sequences is crucial for industrial recommendation and online advertising, yet directly feeding thousands of historical actions into ranking models is computationally prohibitive, while truncation discards long-range signals. Existing lifelong-interest methods retrieve target-relevant behaviors for each candidate, coupling long-sequence modeling with candidate scoring and repeated online cost. Recent target-independent compression methods enable cached user summaries, but often append query tokens at the sequence end and use bidirectional encoding, producing unordered and redundant summaries that overlook temporal structure. We propose ChronicleRec, a pre-train-and-transfer framework that compresses an ultra-long behavior sequence once into a chronologically ordered set of Chronicle Tokens. ChronicleRec applies a recency-aware multi-granularity merge, preserving recent behaviors while coarsening distant history. It then interleaves query tokens with the merged sequence and uses a causal encoder, so each query summarizes only the history before its temporal anchor. A multi-horizon design masks different recent-history windows across parallel branches to learn complementary long-range interests. The compressor is pre-trained with a mask-and-predict objective that reconstructs held-out recent behaviors from compressed older history, aligning historical signals with near-present intent. Since Chronicle Tokens are target-independent, they can be cached per user, decoupling ultra-long sequence modeling from online candidate scoring. Experiments on KuaiRand and Tencent AdLive show that ChronicleRec outperforms recent-window and single-pass compression baselines while approaching full-attention performance. Token analyses reveal temporally organized and complementary representations, and a seven-day online A/B test confirms significant production gains.

Closest papers

Nearest neighbours by meaning across the whole index.

SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching

cs.IR cs.AI ·2026-09-08 ·Lin Guan, Jia-Qi Yang, Zhishan Zhao, Jiaqi Huang, Hangyu Wang +13 more
top venue similarity 0.81
Abstract · RecSys'26 Industry Track, accepted as a long oral presentation. Production deployment on Douyin. Topics: industrial recommender systems, seq…

Modeling long-term user behavior is central to sequential recommendation and billion-scale industrial recommender systems, yet production ranking models operate under strict latency, memory, communication, and training-throughput constraints. At the 100K scale, the challenge extends beyond attention complexity: raw sequence features must be stored, transferred, and repeatedly processed during training and online serving. Existing approaches based on history truncation, multi-stage behavior retrieval, compressed lifelong histories, or train-short/infer-long extrapolation either weaken end-to-end optimization or retain substantial length-dependent cost. We present SequenceO1, an end-to-end framework for ultra-long user behavior sequence modeling, deployed at full traffic on Douyin with histories of up to 100K interactions. SequenceO1 follows a compress-then-reason design. Its Sketch Attention (SA) uses learnable prototypes and prototype-wise normalization to compress the raw history into a fixed-size, target-agnostic user representation. Target-conditioned Stacked Target-to-History Cross Attention (STCA) then models complementary time scales: a recent 10K suffix for short-term interests and the compact sketch for long-term preferences. To make training and inference practical, SequenceO1 combines low-rank user representation caching, multi-request user-level batching, pipeline lift, and a fused FlashSA kernel to amortize feature storage, communication, and computation across targets, training instances, and consecutive requests. Production experiments show consistent offline and online gains, while the compact cached sketch retains most of the benefit of directly scaling end-to-end sequence ranking to 100K. These results provide a practical model-system approach to efficient attention, sequence compression, and scalable long-sequence and long-context recommendation systems.

0.52

MuSeR: Scalable Long-sequence Recommendation with Multi-interest Modeling

cs.IR ·2026-09-20 ·Yongkang Fu, Beining Bao, Yu Jiang, Xiangyu Zhao, Hongyang Wei +8 more
similarity 0.76
Abstract

Ultra-long user behavior sequences carry rich signals of stable and diverse preferences, yet industrial recommender systems typically truncate histories to a few hundred actions under strict latency and memory budgets, leaving long-term interests under-utilized. Users also pursue multiple heterogeneous intents across modalities such as news, Q&A, and short video, which sparse ID embeddings alone struggle to represent. We present Multi-interest Sequence Representation (MuSeR), a retrieval framework built on the deployed MGS system, which integrates three components: (i) hierarchical temporal compression, which retains recent actions at full resolution while progressively pooling older segments, so that per-user histories of $10^{4}$-$10^{5}$ interactions fit within a fixed serving budget; (ii) disentangled multi-query interest extraction with orthogonality regularization; and (iii) multimodal semantic alignment, which augments sparse item IDs with textual summaries distilled from a large language model. For industrial deployment, MuSeR further adopts asynchronous user-representation refresh with adaptive caching and hierarchical beam-search retrieval across heterogeneous hardware. On three public benchmarks and a large-scale industrial dataset, MuSeR consistently improves Recall@$K$ over strong long-sequence and multi-interest baselines. In online A/B tests on Baidu APP's homepage feed, discovery feed, and short-video scenarios, MuSeR yields +0.26% daily active users and +0.89% total session duration (both statistically significant, p<0.05), alongside reduced serving latency and cost. Rather than proposing a new modeling primitive, our contribution is a system-level integration that makes long-term, multi-interest, and multimodal modeling jointly deployable in a real-time production pipeline, together with the engineering practices required to sustain it.

0.05

Query- and Candidate-Aware Long User Sequence Modeling for Sponsored Product Ranking in Search

cs.LG stat.ML ·2026-11-08 ·Dongyue Xie, Sergey Kolbin, Manavender Malgireddy, Iurii Shcherbak, Weixing Tang +5 more
published similarity 0.75
Abstract · Accepted to ACM RecSys 2026 Workshop on Context-Aware Recommender Systems (CARS)

User sequence modeling is widely used in recommendation and advertising. Search ranking presents an additional challenge: the current query expresses immediate intent, while a candidate product determines which parts of a user's long-term history are relevant. We present a production sequence framework for sponsored-product ranking that augments recent behavior with two retrieval channels over long-term history: candidate-nearest-neighbor retrieval and query-nearest-neighbor retrieval. The selected actions are encoded together with dedicated query and candidate context tokens, while candidate-aware early fusion injects the candidate into every history token. Offline ablations show that dual retrieval improves major ranking metrics relative to the production baseline. In an online A/B test on sponsored-product search traffic, the model increased CTR, attributed conversion rate, and orders per impression.

0.23

Dual-Interest Sequential Product Recommendation With Multi-Granular SSM

cs.AI cs.LG ·2026-09-18 ·Shuiying Liao, P. Y. Mok
similarity 0.72
Abstract

Sequential recommendation aims to predict the next item a user will interact with based on their historical behavior. Advances in Transformers have significantly improved sequential recommendation but are still limited by cost efficiency. Although State Space Models (SSMs) have recently enabled efficient long-range modeling, most existing methods encode each item with a single static contextual role, overlooking the phenomenon of item polysemy. In fact, the same item often plays different semantic roles depending on user context, and existing methods are limited in capturing dynamic behavior across different temporal granularities. In this work, we propose DSRec, a novel dual-interest cross-SSM model that explicitly disentangles item roles across long-term and short-term semantic context. Sequential items are encoded into long-term interest embeddings that capture stable preferences via historical aggregation, and a short-term interest branch that emphasizes local session intent modulated by inter-click time intervals. These interest embeddings are processed through distinct SSM encoders: a full-sequence Mamba for long-term modeling, and a time-modulated SSM that dynamically adjusts state evolution based on temporal gaps. To enable effective cross-granularity alignment, we adopt a residual cross-fusion mechanism that exchanges contextual information between the two branches while preserving semantic independence. Experiments on public benchmarks demonstrate that DSRec outperforms other state-of-the-art methods.

0.05

KuaFu: Compressing Long User Behavior into Understanding at Billion Scale

cs.IR cs.CLcs.LG ·2026-09-25 ·Jiahao Hui, Lin Zhu, Ben Tan, Yishen Hu, Jingdong Shu +5 more
similarity 0.72
Abstract · 12 pages, 6 figures, 3 tables

Learning user preferences from long behavioral sequences is fundamental to large-scale advertising, recommender systems, and conversational agents. In industrial systems, preference modeling is typically task-wise, with different downstream tasks independently processing the same user history to capture different aspects of preference. This repeated processing creates substantial redundancy at billion-user scale, especially as preferences evolve and representations require frequent updates. We present KuaFu, a unified user preference modeling system that replaces repeated user-level sequence encoding with reusable item-level compressed representations. Each behavioral item is encoded once and shared across users and downstream tasks, while a user's history is built chronologically and extended with new item representations as they arrive, letting KuaFu capture evolving preferences without re-encoding the entire history. To reduce inference and storage overhead, KuaFu introduces a two-axis projector: token-axis compression shortens behavior sequences by up to 15x and reduces each item from 150 KB to 10 KB, while width-axis compression reduces the cached representation to 0.5 KB. We further develop a four-stage training pipeline consisting of reconstruction pre-training, preference-understanding post-training, joint compression-generation training, and hallucination-aware reinforcement learning, progressively improving representation fidelity and downstream utility. Across six MRQA and five RecBench benchmarks, KuaFu consistently outperforms existing compression methods at comparable ratios, with gains of up to 17.7 EM and 1.90 AUC points respectively. KuaFu has been deployed and evaluated on the Tencent advertising and recommendation platform for ten months. Over this period, it improves per-GPU throughput by up to 350%, saves 190 GPUs, and increases overall GMV by 1.37%.

0.25

ECHO: Efficient Cascade with Homogeneous Operators for Distillation-Guided 100K-Sequence Modeling

cs.IR ·2026-10-20 ·Wenhao Li, Qinglin Jia, Chenghou Jin, Zihong Xian, Jiezhou Ji +7 more
similarity 0.72
Abstract · 12 pages, 8 figures

Modeling lifetime user behavior sequences is essential for capturing long-range preferences in industrial recommendation, yet sequences containing up to 100K interactions make target-aware ranking prohibitively expensive. Search-based methods that retrieve from raw sequences incur candidate-dependent costs that grow with sequence length, whereas latent compression methods reduce computation at the expense of item-level addressability. We propose ECHO (Efficient Cascade with Homogeneous Operators), a framework that balances efficiency and fine-grained retrieval by retaining a subset of original event records per user through a Distilled Selector (DS) and retrieving a smaller set of target-relevant records for each candidate through a Target-aware Fine-grained Retriever (TFR). DS is supervised by a candidate-aware full-history teacher via per-target distillation, enabling target-aware user-level selection without requiring full-history computation for every candidate. The retriever and target-attention scorer share the relevance parameters learned by the teacher, ensuring consistent relevance estimation throughout the cascade. At a sequence length of 100K, ECHO reduces online sequence-branch FLOPs by approximately 160 times compared with a representative end-to-end model. Three deployment optimizations, including common user inputs, compact record encoding, and key-side projection absorption, further reduce memory, storage, and data-movement overhead. Experiments on KuaiRand-1K and an industrial-scale short-video platform demonstrate relative GAUC improvements ranging from 0.113% to 1.520% over state-of-the-art baselines across multiple tasks. Furthermore, online A/B tests in two production scenarios, Kuaishou and Kuaishou Lite, show that ECHO improves app usage time by 0.240% and 0.373%, respectively.

0.24

Closing the Long-Short View Gap in Sequential Recommendation without Cached History

cs.IR ·2026-09-05 ·Lingfeng Shi, Chengkai Huang, Lina Yao, James Caverlee
top venue similarity 0.72
Abstract · Accepted at CIKM 2026

Sequential recommenders are typically trained on long user histories to capture rich behavioral signals, yet serving with training-length sequences is often impractical due to real-time efficiency constraints. Directly using only recent behaviors leads to a severe performance drop. To bridge this gap, existing approaches compress user histories into persistent per-user states, storing and retrieving them at inference time; while effective, they impose non-trivial infrastructure overhead and offer little remedy in cold-start scenarios. In this paper, we empirically identify two structural flaws rooted in geometric properties and dataset sparsity, and propose a novel two-stage framework to close the long-short-view performance gap. Specifically, in the first stage, we replace the commonly used dot-product with angular similarity scoring and leverage a modified softmax to counter prefix position bias. In the second stage, we fine-tune only bias and LayerNorm components, which are universal to standard sequential backbones, for further improvement. Both stages are guided by carefully designed learning objectives. Extensive experiments on two representative backbones across three public datasets demonstrate the effectiveness of our proposed framework.

0.27

BehaveRec: When Does Long-Term Behavioral History Help Sequential Recommendation? A Study with Session, History, and Community Signals

cs.AI cs.IR ·2026-10-26 ·Aysenur Inan, Reza Yousefi Maragheh, Jayesh Uddhav Kudase, Selene Xu, Kaushiki Nag +2 more
hot direction similarity 0.70
Abstract

Sequential recommendation models typically predict a user's next action from recent session interactions, which limits personalization when short-term signals are weak or ambiguous. Rather than assuming that incorporating longer-term history and community signals is universally beneficial, we study when it actually helps. We propose BehaveRec, a three-signal framework combining a causal session transformer, a bidirectional User History Encoder over a 90-day window, and a GraphSAGE module over a user-user graph built from behavioral embedding similarity rather than item co-occurrence; a learned gate blends the three signals per user. On an industrial dataset of 137M sequences from 12.6M users, BehaveRec improves R@5 by +3.2% over SASRec, with a larger-capacity SASRec baseline (more parameters than BehaveRec) still trailing by +2.5%, ruling out capacity as the explanation. More importantly, a user-level stratification by history length reveals a clear dose-response relationship: BehaveRec's advantage is negligible below roughly 10 interactions and grows monotonically with history richness, reaching +0.017 R@5 for users with 101+ events. This pattern is corroborated across one industrial and three public datasets spanning history-rich to history-sparse regimes, where session-only models remain preferable. These headline results, and the resulting deployment threshold, are established for category-level intent prediction; a preliminary, single-run check on the full item-level vocabulary yields a directionally mixed result and is scoped as an open question rather than a claim. We use these results to derive a practical deployment rule -- roughly 20 interactions per user -- for when long-term history and behavioral-graph signals are worth their added complexity. Despite this added complexity, BehaveRec remains lightweight: it uses 5.67M parameters and trains on a single GPU in under two days.

0.24