CS Paper Radar

LatentReRig: An SDF-Based VAE with Dual Decoders for Latent-Space Deformation Conditioning

Daniele Dolci, Fabrizio Poggioni, Carlo Melchiorri

cs.CV cs.GR ·submitted 2026-09-21 21:49 UTC · v1

Abstract

Transferring deformation between characters with different geometry and topology is challenging because conventional rigs encode behaviour through character-specific structures and correspondences. We present LatentReRig, an experimental framework that investigates whether pose-associated changes can instead be represented as reusable directions in a learned geometric latent space. An SDF-based variational autoencoder is coupled with two decoders: one reconstructs the implicit field, while the other predicts target vertex positions from source geometry and latent deformation conditioning. The source geometry may be neutral or already deformed. Experiments on a controlled humanoid dataset show that several poses induce coherent latent directions across identities, particularly for broad articulated motions. These signals can guide deformation of unseen characters, but explicit predictions remain less accurate for localized changes and corrective contributions. Diagnostic comparisons with repeated SDF sampling show that inter-identity distances exceed same-geometry resampling variability on average, while pose signals exhibit different margins above this baseline. The results support the presence of reusable pose-related structure and identify stable local conditioning and accurate mesh decoding as complementary requirements for improving transfer.

Comment
13 pages, 11 figures, extracted from final dissertation

Closest papers

Nearest neighbours by meaning across the whole index.

Mesh-Agnostic Refinement Models for Latent Space Encoding of 3D Human Body Shapes

cs.CV ·2026-11-02 ·Chaima Zouaoui, Jonathan Cerqueira, Sylvain Arguillere, Martin Bauer, Nicolas Charon +2 more
similarity 0.71
Abstract

Parametric human body models such as the Skinned Multi-Person Linear (SMPL) model provide compact representations of human shape and pose, but their reconstructions remain affected by both errors in estimated latent parameters and limitations in the decoder's expressive capacity. We propose a framework for refining these reconstructions by exploiting the geometric discrepancy between an initial decoded surface and a target mesh, without requiring vertex correspondences between them. The framework comprises three complementary approaches. Latent-VaRefine uses varifold distance gradients to correct the estimated latent code while preserving the original parametric representation. Vertex-VaRefine processes multiscale varifold-gradient features with a DiffusionNet backbone to predict vertex displacements in a canonical pose, yielding an approximately pose-equivariant refinement pipeline. ElaOpt directly optimizes surface deformations under elastic regularization and normal direction constraints. On AMASS, Vertex-VaRefine reduces the mean vertex-to-vertex error of OmniFit reconstructions from 1.181 to 0.342 cm, a 71.0 percent reduction, while combining Latent-VaRefine with ElaOpt reduces the asymmetric Chamfer distance of ArtEq reconstructions from 1.681 to 0.717 cm, a 57.3 percent reduction. Evaluations with SMPL-H and SMPL-X demonstrate the framework's adaptability to different body models, while additional experiments on remeshed targets and raw scans assess its robustness to changes in discretization and geometry beyond the parametric model. These results highlight the complementary strengths of latent space correction, learned vertex refinement, and elastic optimization, offering different trade-offs between preserving the model parameterization and improving geometric fidelity.

0.24

ReCHOIR: Contact-guided Human Object Interaction Retargeting to Diverse Characters

cs.GR ·2026-09-10 ·Chaelin Kim, Seokhyeon Hong, Kwan Yun, Soojin Choi, Inseo Jang +1 more
top venuecode/data similarity 0.68
Abstract · SIGGRAPH Asia 2026 (Journal Track). Project page: https://cherry-leki.github.io/projects/ReCHOIR/

We present ReCHOIR, a novel contact-guided motion retargeting method for transferring human object interaction (HOI) motions across diverse humanoid characters. Unlike prior motion retargeting methods that primarily focus on transferring human motion alone, our goal is to preserve not only the semantics of the original body movement but also consistent interaction between the character and the manipulated object, while jointly producing aligned target human and object motions. Given source HOI motion, object geometry, and contact cues extracted from the source interaction, ReCHOIR retargets an HOI sequence to target characters with different skeletal configurations while maintaining both motion semantics and contact-consistent interaction patterns. Our method builds on a Part-Aware Motion Embedding (PAME) autoencoder, which encodes full-body motion into a shared body-part-wise latent space. This representation enables generalization across heterogeneous skeletons while preserving local motion semantics beneficial for part-aware adaptation in HOI retargeting. On top of this representation, we introduce a contact-guided retargeting module and an object motion decoder for HOI retargeting. The contact-guided retargeting module treats the source object interaction as a condition for refining target character motion: object- and contact-related signals are encoded into a body-part-aligned latent representation and injected into decoding through a residual control branch, enabling stronger adaptation in interaction-relevant body regions without discarding the underlying motion prior. In parallel, the object motion decoder predicts a target object motion aligned with the refined target character motion, ensuring that the object trajectory remains consistent with how the interaction is realized by the target character.

0.45

MoRE: Motion-Regularized Encoding for Articulated 3D Generation

cs.CV ·2026-10-21 ·Guantian Zheng, Shan Jin, Tim Tso, Hongfei Cai
similarity 0.68
Abstract · 6 pages, 3 figures, 4 tables

Generating an articulated object from a single image requires recovering both its appearance and the mechanism governing its parts. A compact encoding of joint attributes does not by itself ensure that reconstruction preserves the resulting motion. We present MoRE, Motion-Regularized Encoding for articulated 3D generation. MoRE represents coordinate-aligned joint lines with axis classification and two axis-orthogonal Plücker moment coefficients, removing the arbitrary choice of an origin along the axis. During representation learning, decoded joint hypotheses are used to move corresponding three-dimensional anchors, and their motions are compared with those of annotated joints. A differentiable expectation over joint types and axis directions couples attribute reconstruction to the motion it induces. An image-conditioned flow learns the resulting functional latents, while a multi-state-adapted Gaussian model supplies appearance. At inference, recovered part assignments and joints articulate a single set of rest-pose Gaussians without regenerating the object for each state. Experiments on PartNet-Mobility and ACD show stronger part-box agreement, articulated-state geometry and visual similarity than the compared baselines, while joint prediction reduces axis, origin and range errors relative to ArtiLatent. Controlled ablations support the joint-line encoding and show that motion supervision reduces anchor-motion error. Code and pretrained checkpoints will be released in the future.

0.26

BAL: Bidirectional Autoregression in Latent Space for Learning Human Movement Representations

cs.CV cs.LG ·2026-11-13 ·Genki Kinoshita, Ko Nishino
top venuetheorycode/data similarity 0.67
Abstract · Accepted to NeurIPS 2026. Project page: https://github.com/kyotovision-public/bal

Learning semantically meaningful human movement representations from 3D pose sequences is essential for human behavior modeling. Existing self-supervised methods derive supervision from artificial assumptions, e.g., invariance to augmentation within a certain range or recoverability of masked regions at a certain temporal scale, introducing heuristic hyperparameters that make the learned semantics sensitive to their settings. We propose BAL, a self-supervised framework that instead exploits supervision intrinsic to human movement: the natural ordering structure of everyday behavior. BAL autoregressively predicts in a semantically abstract latent space, analogous to how humans anticipate others' actions rather than their precise joint configurations. Bidirectional autoregressive prediction, both forward and backward in time, fully leverages these ordering constraints while enabling representation extraction using both past and future context at inference. To suppress representation collapse, BAL decomposes each prediction target into a global component capturing sequence-level semantics and a local component capturing context-independent movement. We prove that the local targets retain nontrivial angular diversity unless the representations are completely collapsed, preserving a meaningful training signal. Experiments on large-scale datasets demonstrate that BAL's representations are highly effective for motion captioning, forecasting, and interpolation, outperforming baselines on most evaluation metrics.

0.31

ULIFLOW: Unified Latent Human-Object Interaction Flow

cs.CV ·2026-10-16 ·Wenqi Cai, Chunzhi Gu, Weichen Zhou, Yawen Zou, Yijin Wei +3 more
position similarity 0.67
Abstract

Text-conditioned 3D human--object interaction (HOI) generation remains challenging because individually plausible human and object motions do not necessarily form a coherent interaction. We argue that a key limitation lies in conventional skeleton-based representations, which compactly describe body kinematics while leaving surface-level interaction geometry largely implicit, often requiring additional contact-specific regularizers. We propose ULIFlow, a framework that explicitly models continuous human--object relations within a unified latent space. To this end, we derive continuous marker--object relations from sparse body-surface markers and treat them as an explicit interaction modality alongside human motion and object dynamics. A Mixture-of-Transformers (MoT) autoencoder is then designed to jointly model these heterogeneous streams while preserving their modality-specific structures. Importantly, we let the relation stream directly participate in human and object reconstruction, encouraging the shared latent representation to preserve both individual dynamics and the interaction geometry that coordinates them. To further achieve variable-length generation, we introduce a blockwise Flow Forcing scheme with global sequence-progress conditioning, enabling progressive continuation from fixed-length training windows while maintaining consistent long-horizon dynamics. Extensive experiments demonstrate that ULIFlow consistently outperforms prior work in overall generation quality and human--object coordination, while preserving coherent interactions over requested durations beyond the training horizon.

0.10

MorphoStyle: Motion Style Transfer with Morphology Control

cs.CV ·2026-09-12 ·Xin Feng, Eleonora D'Arnese, Mohan Sridharan
publishedbenchmarkcode/data similarity 0.66
Abstract · Accepted by BMVC2026

Human motion may be viewed as a combination of action content, style, and body morphology. Existing motion style transfer methods transfer a reference style onto a content motion while assuming a canonical body, whereas shape-aware motion generators adapt motion to a target shape without explicit style control. This separation of motion style and shape (morphology) makes it difficult to generate stylized motions for non-canonical bodies; naively combining a style transfer module with a shape-aware generator often leaks action content from the style reference and disrupts shape-consistent kinematics. We present MorphoStyle, a framework for shape-aware motion style transfer that is built on a shape-conditioned Finite-Scalar-Quantization Variational Auto-Encoder (FSQ-VAE). The key contribution is to pose the desired style transfer as modular latent disentanglement comprising: (i) a contrastive style encoder that extracts content-decoupled style embeddings; (ii) a text-guided style-routing mechanism that locates style-relevant joints in a text-motion feature space; and (iii) a manifold preserving style modulator that injects discriminative style embeddings in content features as a temporally-gated low-rank offset. Extensive experiments on benchmark datasets demonstrate that MorphoStyle outperforms competing baselines in terms of both shape control and motion style transfer, while simultaneously providing quantitative shape control. For more details, please see project website: https://github.com/funkdub/MorphoStyle.

0.43

Pose-based Neural Clothing for Animated Characters

cs.GR cs.CV ·2026-11-13 ·Julian N. Heidenreich, Vinicius C. Azevedo, Jakob Buhmann, Lento Manickathan, Arnold Moon +3 more
published similarity 0.65
Abstract · 19 pages, 18 figures, 5 tables. Includes appendix. Accepted to the ACM SIGGRAPH / Eurographics Symposium on Computer Animation (SCA 2026); t…

Traditional garment simulation for animated characters is a time-consuming bottleneck for artists due to slow simulations and tedious parameter tunings.Recent data-driven methods offer a powerful alternative, capable of learning from a dataset of simulated data to generate high-quality deformations. A lightweight network design allows data-driven systems to achieve a real-time inference speed. This significantly improves the artist workflow, as they can interactively pose a character and see high-quality folds and wrinkles at animation time, thereby allowing to make creative decisions ahead of time. However, existing neural clothing methods have presented appealing results on datasets based on realistic human models (\eg SMPL), but when trained on animated character datasets, they struggle to generalize to unseen poses. In this paper, we aim at designing neural networks and training strategies to generalize better to unseen poses in real-time to enable the interactive posing applications. We focus on a production-oriented setting where the model is trained for a single garment in a quasi-static setup. Technically, we leverage a smooth, expressive latent space by first training a Variational Auto-Encoder (VAE). We then progressively train the VAE encoder to transition from reconstruction to pose-driven prediction. To guide this prediction, we condition the network on both low-dimensional pose parameters and character body surface displacements. Finally, we introduce a novel local gradient histogram loss that mitigates the over-smoothing effect of reconstruction losses. Extensive evaluations on production-level assets and public datasets demonstrate that our method produces sharp, realistic wrinkles, smooth large-scale drape, and temporal coherence in real-time, offering a robust and easily integrable solution for interactive posing workflows.

0.17

Deep Prior Learning for Embodied Perception

cs.CV cs.RO ·2026-10-04 ·Yimou Wu, Jiaxin Guo, Yun-hui Liu, Zheng Li
similarity 0.65
Abstract

Embodied systems need geometric perception that exploits available observations beyond images alone. Recent feed-forward 3D models incorporate geometric priors, including camera poses, intrinsics, and depth. However, handling noisy poses, preserving accurate priors, and recovering physical scale require more than simply accepting these inputs. We introduce \emph{Vision-Prior Geometry Grounded Transformer} (VPGGT), a VGGT-based framework that extends OmniVGGT for prior-aware embodied perception. We formulate sensor-motivated pose corruptions from ground-truth trajectories for training and introduce a parameter-free \emph{prior residual connection} (PRC) to mitigate \emph{prior dilution}, where predictions are less accurate than their supplied pose priors. Our noise formulation targets camera poses; supplied intrinsics and depth receive no additional corruption. We further introduce \emph{Metric Global Attention}, which conditions a global scale token on available pose and depth scales and predicts a shared metric scaling factor for the geometric outputs. Experiments across four datasets show that \emph{PRC} improves translation-direction accuracy and joint pose AUC over a matched training baseline when camera priors are provided for all views, under both exact and corrupted poses. These results support explicit prior access during refinement as a useful addition to feature-level conditioning.

0.20