CS Paper Radar

Stable Signer V3: 4D Deformable Primitive Splatting Is All You Need for Sign Language Production

Sen Fang, Liwei Che, Dimitris N. Metaxas

cs.CV cs.AIcs.CLcs.GR ·submitted 2026-10-18 00:28 UTC · v2
code/data

Abstract

We present Stable Signer V3, an interactive high-resolution sign language avatar system built on 4D Deformable Primitive Splatting. It addresses three limitations of existing avatars: restricted facial expression, inaccurate identity-specific body deformation, and loss of fine appearance details. Shared Face Primitive Splatting adds transferable superquadric primitives for richer expression control; Test-Time Body Primitive Splatting learns an identity-specific virtual muscle layer for more faithful motion; and Test-Time Tunneling Super-Resolution uses TunnelingNet to recover identity-specific high-resolution details. Both test-time modules are optimized once per signer identity; inference then runs entirely from the stored identity parameters. Together, they reduce facial geometry error by 71.4%, body-boundary error by 35.1% across ten identities, and high-resolution RGB reconstruction error by 24.2% on held-out evaluations, while retaining interactive streaming.

Comment
12 figures, 10 tables. Project Page at: https://signerx.github.io/Stable-Signer-V3

Closest papers

Nearest neighbours by meaning across the whole index.

PHOSA: Photorealistic 3D Sign Avatar Modeling and Benchmark

cs.CV ·2026-09-24 ·Haodong Wang, Hezhen Hu, Wengang Zhou, Houqiang Li
top venuebenchmarkcode/data similarity 0.67
Abstract · ECCV 2026, project page: https://naaapi.github.io/PHOSA

In this work, we focus on photorealistic sign avatar modeling, which is crucial for effective communication with the Deaf community and is characterized by complex hand gestures and nuanced facial expressions. To this end, we introduce MVSign, the first multi-view Chinese sign language dataset co-designed with Deaf experts, featuring diverse gestures and rich annotations. For precise SMPL-X annotation, we develop a hybrid fitting pipeline that produces accurate body, hand, and facial parameters and can also be applied to the monocular setting. Building on MVSign, we propose a decoupled sign avatar representation that isolates body, head, and hand components to capture complex articulations, together with a motion-aware sampling strategy to handle motion blur and balance gesture diversity. Extensive experiments demonstrate that our method achieves high-fidelity visual results on MVSign, particularly in detailed hand and facial regions, and generalizes well to in-the-wild monocular sign language videos. Project page: https://naaapi.github.io/PHOSA.

0.40

SignRefine: Adapting Foundational Video Models for Sign Language Generation

cs.CV ·2026-09-08 ·Anton Pelykh, Edward Fish, Ozge Mercanoglu Sincan, Richard Bowden
top venuedataset similarity 0.65
Abstract

Sign language video generation demands precise hand and facial articulation, yet modern video diffusion models, trained predominantly on spoken-language video, produce artifacts that render signing unintelligible. We propose SignRefine, a sign language video generation model that produces comprehensible signing from 2D keypoint conditioning alone, generalizing across appearances and visual conditions. Our approach builds on a pretrained video diffusion transformer and introduces local adapters with spatial grounding to selectively refine hand and face regions, steering the strong base model's prior toward accurate articulation. To enable this work and support broader sign language research, we present NVSign, a large-scale dataset of video content natively produced in sign language, offering diverse signer appearances, environments, and natural conversational settings. Trained on this data, our model shows up to 30% improvement in hand pose precision metrics over the strongest baseline and is preferred by sign language users for visual quality and comprehensibility in more than 80% of comparisons.

0.38

SignGPT: Toward LLM-Mediated Sign Language Interaction through Gloss-Free Translation and Generation

cs.CV ·2026-09-18 ·Ronghui Li, Jun Dong, Zhongyuan Hu, Zunnan Xu, Jun Zhou +6 more
benchmark similarity 0.63
Abstract

Large language models (LLMs) provide limited support for sign language interaction. Unifying sign language translation (SLT) and generation (SLG) to enable sign language as both input and output can reduce switching between separate models during sign-text interaction. We present SignGPT, a unified, pose-based framework for gloss-free SLT and SLG. SignGPT integrates part-aware hierarchical representations of body, hand, and facial motion into a shared language model and employs asymmetric multi-token prediction and progressive training for bidirectional modeling. We evaluate SignGPT on How2Sign (ASL) and Phoenix-2014T (DGS) through benchmark comparisons, qualitative analyses, and component ablations. An exploratory study with 12 Deaf ASL signers assesses an LLM-mediated sign-to-sign response pipeline, highlighting the potential of unified modeling to support sign language conversation (SLC).

0.06

RoboSTAR: Next-Scale Autoregressive Sign Language Translation for Humanoid Robots

cs.CV ·2026-09-26 ·Yujia Zeng, Chensheng Peng, Yuxin Chen, Alex Shao, Nathan Jew +1 more
similarity 0.61
Abstract

Sign-language interpretation in public communication relies on qualified professional interpreters and can be difficult to scale, motivating robotic signing as a complementary accessibility interface. We present RoBoSTAR, a text-conditioned sign language production (SLP) framework for generating human-centric sign motion that can be retargeted for robotic execution, with speech supported optionally through an external ASR front end. Conventional autoregressive approaches flatten motion into a single full-resolution token sequence, forcing long-range and local dependencies to be modeled at a uniform temporal granularity. RoBoSTAR instead combines part-wise Finite Scalar Quantization with next-scale autoregression, generating motion over progressively finer temporal resolutions while predicting synchronized body and hand tokens in parallel within each step. This coarse-to-fine formulation provides compact long-range context before progressively refining motion details, while self-conditioning and context corruption improve robustness to cross-scale prediction errors. The generated motion is subsequently retargeted for physical humanoid execution. Extensive qualitative and quantitative evaluations are conducted to demonstrate the effectiveness of RoBoSTAR.

0.15

Not All Challenges Are Equal: Evaluating Interactive Liveness Detection Against Real-Time Face Swaps

cs.CV cs.CR ·2026-10-12 ·Anton Firc, David Drtil, Martin Perešíni, Milan Šalko, Vojtěch Staněk +1 more
publishedhot direction similarity 0.61
Abstract · Accepted at BIOSIG 2026

Remote identity verification assumes that a camera stream shows a live person, yet public real-time face-swap tools can replace that person before the stream reaches the verifier. Passive deepfake detectors generalize poorly to new generators, while interactive challenges are often selected without evidence of which gestures expose current attacks. We evaluate randomized gesture challenges in 108 live sessions with six subjects, 24 impersonated identities, and two face-swap tools, DeepFaceLive and FaceFusion, injected through a virtual camera. The static calibration segment reaches a session-level AUC of 0.878 with a frozen UCF spatial detector, while the mean across the active-gesture windows reaches 0.988. Pose and combined gestures have the highest point estimates, whereas expressions show no clear improvement. Hand occlusion against FaceFusion, configured with an occlusion mask, scores below the static baseline. The end-to-end time required to satisfy the validator also distinguishes attacks from genuine sessions, achieving a session-level AUC of 0.732, although this signal includes synthesis and transport delays. These results show that challenge sets should be tested against current attack configurations and that response time can provide an additional, deployment-specific signal.

0.62

MySign: A High-Fidelity Motion-Capture Dataset for 3D Sign Generation in Bahasa Isyarat Malaysia

cs.CV ·2026-11-08 ·Jiayu Shen, Kalin Stefanov, Lay-Ki Soon, Vee Yee Chong, Alpha Agape Gopalai +1 more
benchmarkdatasetcode/data similarity 0.61
Abstract

Progress in 3D sign generation is constrained by the scarcity of high-fidelity 3D sign language datasets, a limitation that is particularly severe for under-resourced languages. We introduce MySign, the first motion-capture dataset for isolated signs in Bahasa Isyarat Malaysia (BIM). Its 5,000 samples cover 1,000 lexical items (glosses) and were produced by five Deaf native signers. Prior datasets that rely on SMPL-X parameters regressed from video provide pseudo-labels, which are often affected by depth ambiguity, inaccurate finger poses, and temporal jitter. MySign instead provides ground-truth SMPL-X sequences retargeted from recordings made with a motion-capture setup tailored for sign language; these sequences offer higher-fidelity motion, particularly in fine-grained hand and finger articulation. A controlled comparison on the same glosses and signers indicates that training on these sequences improves generation over training on such pseudo-labels. We establish MySign as a challenging benchmark for gloss-conditioned 3D sign generation: state-of-the-art models produce visually plausible outputs but fall short of the ground truth, most notably in fine-grained articulation, revealing persistent gaps in modeling articulated sign motion. In contrast, recognition baselines consistently achieve high lexical accuracy, indicating that the dataset reliably supports supervised learning. The dataset is available at https://huggingface.co/datasets/mysigner/MySign

0.33

SignMimic: Robust High-Quality Sign Language Motion Generation via Human-Shape-Oblivious Pose Transfer Guidance

cs.CV ·2026-09-12 ·Zhewen He, Junyi Yu, Haomian Huang, Zhenhua Li, Yi Fang
similarity 0.59
Abstract

We study the challenge of sign language video mimicking: given a driving video and a single reference frame, synthesize a video where the target signer reproduces the source motion while preserving identity and linguistic form. Prior pipelines entangle rigid motion, non-rigid deformation, and view-dependent completion in a monolithic generator, causing handshape drift and spatio-temporal instability. We present SignMimic, which (i) applies a TNet-based model to study SE(3) rigid canonicalization to stabilize global pose, (ii) performs non-rigid adaptation in a canonical space to preserve fine-grained articulators (hands/face) and coarticulation via NIF2D, and (iii) uses Pose-MAE-style completion before conditional video diffusion. This factorization injects geometric and linguistic priors, yielding shape and spatio-temporal consistency. On several large-scale datasets (ASL 50K, How2Sign, CSL News), SignMimic achieves state-of-the-art-level performance on video quality, identity similarity, and frame continuity while also achieving minimal loss when performing back translation (SLT) on generated videos. Ablations confirm the role of rigid canonicalization, non-rigid adaptation, and completion. Code is available at https://anonymous.4open.science/r/UniSignMimicTurbo-6088; model checkpoints and video examples will be released.

0.19

SignRR: Retrieve and Refine Real Motion for Sign Language Production

cs.CV ·2026-08-28 ·Fidel Omar Tito Cruz, Angie Sanchez Marquina, Summy Farfan, Gissella Bejarano
published similarity 0.56
Abstract · Accepted at the 37th British Machine Vision Conference (BMVC 2026)

Sign language production (SLP) aims to generate continuous signing motion from spoken language, often through gloss-to-pose generation. Prior work mainly follows two paradigms. Generative models synthesize motion from a learned prior or from noise, without reference to an observed signing instance, making rare hand configurations and signer-specific articulation difficult to preserve. Retrieval-based methods reuse real, well-articulated motion segments, but concatenating segments from different signers and co-articulation contexts can introduce rhythm and style inconsistencies across the full sequence, not only at segment boundaries. These limitations suggest a complementary solution: use retrieval to provide realistic articulation, and use learned refinement to impose the global coherence that retrieval alone lacks. We therefore propose retrieve-and-refine, a paradigm that starts from real retrieved motion and refines it into a globally coherent signing sequence rather than generating motion from scratch. Our framework, SignRR, initializes motion from a dictionary of real sign segments and refines the full sequence with a part-aware Residual VQ-VAE, where residual quantization preserves fine hand articulation and temporal length differences are handled in the latent space. Experiments on PHOENIX14T and CSL-Daily show that SignRR achieves state-of-the-art back-translation performance while maintaining competitive pose quality.

0.22