CS Paper Radar

Multi-Person Human Motion Forecasting in Complex Scenes

Serdar Ozsoy, Lars Doorenbos, Juergen Gall

cs.CV cs.AI ·submitted 2026-08-27 12:25 UTC · v1
published

Abstract

Accurately forecasting the movement of people in complex scenes requires reasoning over the past and present state of the entire environment. In this context, effectively incorporating object information and social interactions into a unified framework remains particularly challenging. To address this, we propose Object-Conditioned Social Diffusion (OCSD), a conditional diffusion model that integrates motion history, multi-person interactions, and object cues into a single framework. OCSD uses an object-conditioning mechanism that modulates denoising at every timestep, enabling fine-grained human-object reasoning, and a social encoder that models the interactions between all humans in the scene. As a result, our model naturally handles varying group sizes, complex social interactions, and supports sampling multiple plausible futures. Extensive experiments show that OCSD achieves state-of-the-art results on the Humans in Kitchens (HiK) and HOI-M3 benchmarks. It reduces the two-second path error by 121.5 mm (31.3%) on HiK and 130.5 mm (33.2%) on HOI-M3 compared to prior work, and produces more realistic long-term forecasts.

Comment
Accepted to GCPR 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Socialality Anchors: Towards Group-bounded Trajectory Prediction

cs.CV ·2026-09-29 ·Ziqian Zou, Conghao Wong, Qinmu Peng, Xinge You
top venue similarity 0.71
Abstract

Trajectory prediction is a key component for understanding human behavior patterns in dynamic scenes. Researchers have devoted substantial efforts to modeling social interactions, especially group-wise interactions, since group membership often reflects shared intention, coordinated motion, and stable mutual adaptation, thus providing a persistent and semantically meaningful social prior for forecasting. However, existing group modeling methods may rely on a fixed threshold and infer groups mainly from agents' relative positions within the observation window, overlooking the fact that grouping rules should be agent-specific, temporally coherent, and context-adaptive across diverse personalities, culturalities, and evolving interaction contexts. Inspired by human social perception that alternates between interpersonal distance in boundary-sensitive situations and relative speed consistency in dynamic interactions, we propose Socialality, a human-inspired trajectory prediction framework with interpretable Socialality anchors and an extended grouping window for stable, context-aware grouping inference. Concretely, Socialality introduces a duo-scalar-controlled grouping kernel Socialality that jointly leverages historical observations and short-term future trajectory previews to learn agent-specific grouping rules, and employs a group-wise perception mechanism to model in-group and out-of-group interactions in an intuitive and explainable manner. Furthermore, we conduct extensive experiments on standard benchmarks to demonstrate the performance gains of Socialality, and provide qualitative analyses and statistical studies of anchor distributions to verify the interpretability and stability of the proposed Socialality anchors.

0.44

FlowMoP: Stochastic Multi-Person Motion Prediction

cs.CV cs.LG ·2026-11-06 ·Aadya Agrawal, Ho Kei Cheng, Alexander Schwing
top venue similarity 0.68
Abstract · Accepted at NeurIPS 2026. 22 pages, including references and appendix

Predicting the future motion of multiple people is inherently stochastic: the same observed history admits many plausible continuations depending on intent, coordination, and social context. Yet the dominant paradigm in multi-person 3D motion prediction remains deterministic, with existing methods producing a fixed number of outputs per agent. In contrast, we present a conditional flow matching framework that jointly generates diverse future motions for multiple agents, modeling both global trajectories and root-relative poses. A B-spline reparameterization compresses the generative state space while enforcing temporal smoothness, and a dual-stream transformer conditions each agent's predictions on observed neighbor motion for interaction-aware forecasting. Our method achieves state-of-the-art performance on CMU-Mocap (UMPM) and 3DPW, outperforming deterministic baselines on joint position error, pose error, and final displacement error despite producing a full predictive distribution. We additionally establish the first motion prediction baseline on WorldPose, a large-scale professional soccer dataset, where social conditioning demonstrably concentrates predictive uncertainty along interaction-constrained directions.

0.35

Adaptive Interaction Network for Human Motion Prediction During Human-Robot Collaboration

cs.CV ·2026-10-12 ·Mengyuan Liu, Yangting Lin, Qiongjie Cui
publishedcode/data similarity 0.66
Abstract · 12 pages, 10 figures, 10 tables. Author accepted manuscript. Published in IEEE Transactions on Image Processing. Code: https://github.com/Ly…

Human motion prediction during human-robot collaboration is critical for achieving safe and efficient interactions in shared environments. Unlike previous studies that have primarily focused on humans and objects, we focus on the robot-aware human motion prediction task, which explicitly models the influence of robots on human motion. This task presents new challenges, such as capturing human-robot heterogeneity and modeling complex spatial interactions. To address these issues, we develop an adaptive interaction network (AINet) model that consists of two branches: a primary branch that predicts future human motion and an auxiliary branch that estimates robot trajectories. The two branches are jointly optimized and coupled via a Local-Global Spatial Interaction (LGSI) Module, which effectively captures fine-grained and global contextual dependencies between human and robot motion sequences. Then, we introduce an Adaptive Weighted Aggregation (AWA) Module to dynamically fuse motion features using context-dependent weights, thereby increasing adaptability across diverse scenarios. Furthermore, we adopt a coarse-to-fine prediction strategy, in which a coarse pseudo-trajectory is first predicted, followed by the refinement of detailed human poses based on that trajectory. Extensive experiments on three challenging datasets reveal that our proposed method achieves superior performance, validating its effectiveness in human-robot collaboration scenarios. Our code is available at: https://github.com/LyTingHub/AINet.

0.47

Controllable Crowd Generation through World-Model Planning

cs.CV ·2026-10-07 ·JunGyu Lee, Jisu Shin, Seunghyun Shin, Hae-Gon Jeon
code/data similarity 0.66
Abstract · 28 pages, 6 figures. Project page: https://jungyu0413.github.io/Ctrl-CWM

Crowd simulation plays a central role in robot navigation, autonomous driving, and urban planning. For these applications, realistic simulation requires crowds to adapt their behavior to environmental changes and user objectives. However, existing methods that rely on predefined control settings have limited flexibility in accommodating new user-specified objectives. To address this limitation, we propose Ctrl-CWM, a multi-agent Controllable Crowd World Model that integrates crowd generation and run-time control. Our key idea is to adapt the world-model principle of planning using imagined futures to crowd simulation. To this end, Ctrl-CWM consists of an encoder that learns a representation of human motion dynamics, an actor that proposes pedestrian displacements, a critic that evaluates imagined crowd trajectories, and a planner that selects actions. We first learn human motion dynamics through trajectory prediction on real-world pedestrian videos and then freeze the encoder to preserve them. Using this representation, the actor generates imagined crowd trajectories through repeated state updates, and the planner combines the critic's scores with user costs to select actions. Repeated planning advances the simulated crowd, while additional user costs introduce new control objectives without retraining. We extensively evaluate crowd generation under varied agent arrival conditions and run-time control across avoidance and attraction scenarios. Ctrl-CWM outperforms the state-of-the-art method on most crowd realism and collision metrics, and adapts crowd behaviors to user-specified objectives introduced during simulation. The project page is available at https://jungyu0413.github.io/Ctrl-CWM

0.34

GEAR: From Dynamic Encoding to Dynamic Activation in Social Trajectory Prediction

cs.CV cs.AI ·2026-09-12 ·Jiaheng Chen, Jiaxing Li, Leixia Wang, Jianan Ju, Tinghe Zhang
top venuepositioncode/data similarity 0.66
Abstract · Accepted by ICDM 2026

Human trajectory prediction requires modeling both individual motion patterns and social interactions among agents. Existing methods have made substantial progress by using attention mechanisms, graph structures, and temporal encoders to capture dynamic social context. However, most of them primarily focus on how social information is encoded, while paying less explicit attention to how the encoded social context should take effect during future trajectory generation. In this paper, we argue that dynamic social encoding does not necessarily imply dynamic social activation. The same interaction context may require different activation strengths across future horizons and scene densities: social cues should be strengthened when interaction evidence is strong, but suppressed when they are weak or noisy. To address this issue, we propose GEAR, a generation-aware bias activation model for human trajectory prediction. Built upon a bias-decomposed trajectory generation formulation, GEAR dynamically activates the individual-motion and social-resonance bias terms at each future step before final trajectory composition. This allows the model to explicitly control when and how strongly individual and social bias components participate in generation. Experiments on ETH-UCY, SDD, and NBA show that GEAR consistently improves the resonance-based baseline and achieves competitive state-of-the-art performance. Further analyses of activation patterns and density-grouped errors validate the importance of calibrating encoded social context during trajectory generation. Our code is available at https://github.com/11isnotavailable/GEAR.git.

0.57

PRISM: Predictive Representation of Interaction Style and Motion for Social Robot Navigation

cs.CV cs.RO ·2026-09-16 ·Bo-Han Chen, Hiromu Taketsugu, Norimichi Ukita
published similarity 0.64
Abstract · ECCV 2026 Workshop on Agent in World

Humans often observe others before interacting and adjust their behavior accordingly. Robot navigation in crowds, however, often represents pedestrians mainly by observed geometric states, leaving individual differences in interaction tendencies implicit. We propose PRISM (Predictive Representation of Interaction Style and Motion), a framework that infers interaction traits from passive observations of human-human interactions. PRISM encodes human trajectories into a continuous ordinal latent space with a transformer encoder trained by Rank-N-Contrast loss, and pairs each inferred trait with a temporal-stability score supplied to the navigation policy. In randomized crowd simulations, PRISM reduces collision rates over the geometry-only baseline and yields small improvements in navigation-time and path-length metrics. These results suggest the utility of passive latent-trait inference for social navigation in dynamic crowds.

0.40

Social Graph Mamba: Forecasting Pedestrian Movements Based on Social Context

cs.RO ·2026-08-11 ·Hong-Son Nguyen, Yen-Chen Liu
similarity 0.64
Abstract · 8 pages, 4 figures

Forecasting pedestrian motion has always been fundamental for autonomous navigation in crowded environments. While attention-based methods achieve strong performance, they suffer from quadratic computational complexity in modeling social interactions, limiting scalability. Additionally, the existing methods often achieve high accuracy on prediction benchmarks at the individual level, but fail to fully capture the natural movement behaviors of crowds in real-world scenarios, particularly group structures. In this study, we propose Social Graph Mamba (SGM), a novel architecture that replaces attention-based social reasoning with Selective State Space Models (SSMs) operating on dynamically constructed interaction graphs. SGM introduces a dynamic interaction graph with social triplet factorization to decompose crowd interactions sequentially, and a community-aware module to effectively discover group structures via differentiable MinCut optimization and conditions both the embedding space and multi-modal decoder on group membership. Our experiments on standard benchmarks (ETH/UCY, SDD) demonstrate competitive performance with linear sequence complexity compared to quadratic attention-based methods. We further validate SGM in physical robot experiments by integrating predicted trajectories into a Social Force Model (SFM) for real-world implementation.

0.10

Generative Interactions: Weaving Multiparty Human Motion with Bilevel Latent Dynamics

cs.AI cs.LG ·2026-09-29 ·Ojas Shirekar, Yash Surange, Augustinas Jučas, Chirag Raman
similarity 0.64
Abstract

Human social behaviour is not a collection of independent motions, but a jointly organised process in which group dynamics and individual variation continuously shape one another. Yet existing social motion models often prioritise plausible trajectories while leaving interaction state implicit, limiting their ability to transfer across groups, tasks, and partial-observation regimes. To address this gap, we introduce Bilevel Representations for Agent Interaction Dynamics (BRAID), a hierarchical sequential latent-variable model for generative multi-person interaction. BRAID explicitly formulates social motion generation as a meta-transfer learning problem: shared interaction priors are learned across datasets and adapted through arbitrary context sets of observed people and joints. The model represents each scene through a group-level latent state that captures shared interaction dynamics and person-level latent states that capture individual behaviour conditioned on the evolving group context. This modelling choice enables coherent generation under full, sparse, or partial observations while exposing compact social-state vectors that can serve as an interface for downstream embodied-agent systems. We evaluate BRAID under a unified SMPL-based representation on social forecasting, tracking and in-filling, and response generation, using metrics that assess not only reconstruction accuracy but also realism, diversity, temporal alignment, and interpersonal coordination. We further analyse the hierarchical latent space, showing that it captures separable group- and individual-level structure.

0.25