CS Paper Radar

Lambda-Hold Control: Human-Like Movement Emerges from a Minimal Task Reward in Predictive Musculoskeletal Simulation

Jun Hyuk Lee, Chihyeong Lee, Jooeun Ahn

cs.RO cs.GRcs.LG ·submitted 2026-08-17 18:28 UTC · v2
code/datahot direction

Abstract

The massive overactuation in the human musculoskeletal system makes it challenging to train musculoskeletal models to generate human-like motion via reinforcement learning, primarily because exploration in the resulting high-dimensional and redundant action space is extremely inefficient. To address this problem, we propose the $λ$-hold controller, inspired by the equilibrium-point (EP) hypothesis, which has been widely supported by extensive evidence from human motor control studies. The policy's control variable is the per-muscle EP threshold length $λ$, from which a stretch-reflex recruitment law computes the muscle excitations automatically. Holding each $λ$ over an interval of the gait phase also sharply reduces the frequency at which the policy must be queried. Consequently, the controller, to our knowledge for the first time, enables a muscle-actuated skeletal model to learn human-like sprinting using only a minimal reward within an hour of training. The efficient exploration through the proposed $λ$-hold controller is not merely an engineering trick but an approach grounded in physiology, bringing together the EP hypothesis, intermittent control, and optimal feedback control. Beyond encapsulating human-like behavior in predictive simulation, this achievement contributes to developing a learnable model of the human motor controller.

Comment
20 pages, 8 figures, 1 table. Project page and video demos: https://lee-jun-hyuk-37.github.io/projects/lambda-hold/. Code: https://github.com/Lee-Jun-Hyuk-37/Lambda-Hold (archived at https://doi.org/10.5281/zenodo.23265020)

Closest papers

Nearest neighbours by meaning across the whole index.

Reflex-Informed Neuromuscular Reinforcement Learning for Muscle-Driven Locomotion

cs.RO cs.GRcs.LG ·2026-09-10 ·Jian Zhou, Xingyu Zhang, Rui Ma, Yu Cao, Shane Xie +1 more
similarity 0.73
Abstract · 28 pages, 13 figures, and 9 tables

Muscle-driven locomotion provides a physically grounded approach to generating realistic human movement. However, achieving both physiological plausibility and adaptability to changes in musculoskeletal capacity and external disturbances remains a fundamental challenge. To address this limitation, we propose a Reflex-Informed Neuromuscular Reinforcement Learning framework for muscle-driven locomotion. Within this framework, a fixed phase-dependent reflex controller serves as the underlying neuromuscular control mechanism, while the reinforcement learning policy produces four biomechanically meaningful residual parameters to modulate key reflex gains and thresholds associated with hip swing, knee support, and ankle propulsion according to the current state. Experimental results demonstrate that the proposed framework generates physiologically plausible locomotion with improved kinematic accuracy and dynamic consistency, as well as better bilateral symmetry and stride-to-stride consistency under nominal walking conditions. The learned policy remains robust under muscle weakness and external perturbations without retraining.

0.47

MSK-Bench: Benchmarking Full-Body Musculoskeletal Motor Control Across Tasks, Control Paradigms, and Physiological Metrics

cs.RO ·2026-09-22 ·Mengtao Ou, Zongzheng Zhang, Zhenghao Xiao, Yixuan Pan, Ziwen Zhuang +5 more
benchmarkcode/data similarity 0.69
Abstract · Project page: https://zzongzheng0918.github.io/MSK-Bench/

Musculoskeletal (MSK) humanoids provide a physiologically grounded embodiment for studying full-body motor control, but their high-dimensional muscle actuation, delayed activation dynamics, and redundant muscle--tendon structures make learning substantially harder than torque-driven humanoid control. Existing MSK benchmarks remain fragmented across gait, prosthetics, dexterous hands, or challenge-specific tracks, leaving full-body muscle-actuated control insufficiently evaluated under standardized tasks, methods, and metrics. We introduce MSK-Bench, a benchmark of 22 full-body motor-control tasks organized into three progressively challenging categories: postural stabilization, common locomotor behaviors, and contact-rich environmental interaction. Under unified task protocols and robustness perturbations, MSK-Bench evaluates 5 representative control paradigms, including reward-based RL, agentic reward tuning, latent-action RL, imitation-prior control, and residual adaptation over imitation priors. Beyond task success and reward, MSK-Bench further reports robustness analysis and physiology-oriented diagnostics, including activation cost, joint smoothness, and EMG-envelope similarity. Our empirical study shows that embodiment-aware exploration and structured action representations improve task coverage in high-dimensional muscle spaces, imitation priors enhance reference-compatible stabilization and locomotion but degrade under contact-rich terrain mismatch, and residual adaptation can recover successful behaviors when fixed references fail. We further find that improved task success does not necessarily imply improved physiological agreement, highlighting the importance of evaluating task performance, robustness, and physiological behavior jointly. MSK-Bench provides a task--method--metric testbed for full-body muscle-actuated humanoid control.

0.34

A generalist policy for muscle-driven control across individual musculoskeletal models

cs.RO ·2026-10-27 ·Bozhao Wang, Yinglei Zhu, Zhengqing Li, Zhiyuan Zhang, Weiming Kang +5 more
similarity 0.68
Abstract · 17 pages, 7 figures. Project website: https://woodater-thu.github.io/MyoAcross/ Bozhao Wang and Yinglei Zhu contributed equally to this work

Differences in musculoskeletal morphology challenge policy reuse across individual models. Here we present MyoAcross, a framework that combines individual musculoskeletal model construction, morphology-conditioned policy learning and virtual model augmentation to enable muscle-driven reference-motion tracking without individual fine-tuning. Across 31 held-out individual models, MyoAcross achieves 99.24% completion and reduces tracking error by 22.60% relative to an unconditioned multilayer perceptron. Policy reuse extends to models of 22 held-out individuals constructed from single-image shape estimates, measured height and body mass, and, with asymmetry augmentation, to models with unilateral lower-limb shortening. Comparisons with measurements from 120 additional individuals show agreement in vertical ground reaction force waveforms and channel-dependent agreement between simulated muscle activations and electromyography. These results establish a common control framework for comparative simulations of movement across individual musculoskeletal models.

0.38

Learning Realistic Athletic Sprinting Without Demonstrations

cs.GR ·2026-09-10 ·William Wang, Nicholas Bianco, Guy Tevet, Jennifer Hicks, C. Karen Liu +2 more
published similarity 0.68
Abstract · 17 pages, 23 figures

We present a muscle-driven simulation system for generating biomechanically accurate motion for high-speed athletic locomotion tasks that does not require motion demonstrations. Our approach integrates state-of-the-art biomechanical athlete models into a new, high-performance GPU simulator capable of running at 1000x real-time. High-throughput simulation enables large-batch reinforcement learning to train control policies that operate directly in the model's high-dimensional muscle excitation space, and are guided only by task-specific episode termination conditions and a reward that encourages maximizing speed while reducing forces needed to respect joint limits. These policies train within a few hours on a single GPU and generate "near visually realistic" motions for complete athletic activities such as a full 100-meter sprint or performing popular athletic locomotion drills like side-shuffling, backpedaling, and carioca. The generated sprinting motions also exhibit strong agreement with experimental data captured from sprinters.

0.62

Burden, Bounty, or Bliss? Muscle-Aware Retargeting and Null-Space Exploration for Whole-Body Musculoskeletal Control

cs.RO ·2026-11-07 ·Ruiyang Kong, Takeshi Oishi
benchmark similarity 0.67
Abstract · 9 pages, 4 figures, 4 tables. Submitted to IEEE ICRA 2027

Learning whole-body musculoskeletal control from human motion is important for developing human-like robots, assistive devices, and rehabilitation systems. This task presents two main challenges, namely reference trajectories that are kinematically plausible but difficult to reproduce through muscle-driven dynamics, and a highly redundant 700-dimensional action space in which unstructured exploration can easily disrupt the target motion. In this paper, we propose a motion-to-policy pipeline that exploits muscle redundancy in two complementary stages. First, muscle-aware retargeting (MAR) performs bounded adaptation of kinematic trajectories subject to hard semantic, anatomical, and contact constraints. We motivate the constraint bounds by analyzing how reference-trajectory artifacts affect muscle-driven tracking. Second, null-space exploration (NSE) estimates a one-control-frame mapping from muscle excitations to generalized forces and injects bounded, anatomically structured perturbations along its null space into the actions of a value-guided flow policy. On a 24-clip motion-capture benchmark, MAR and NSE together solve every reference, reach the full horizon with half the samples of the baseline pipeline, and lower joint-center error by 44% without late collapse. To the best of our knowledge, this is the first end-to-end pipeline that transforms monocular video into a trained whole-body imitation policy controlling 700 muscles.

0.13

HULK: Learning Whole-Body Forceful Loco-Manipulation for Humanoids

cs.RO cs.LG ·2026-10-06 ·An Dang, Arturo Flores Alvarez, Yu-Ming Chen, Conor Mc Gartoll, Helen Sun +3 more
similarity 0.67
Abstract · 16 pages, 7 figures, IEEE International Conference on Robotics and Automation 2027

Humanoid loco-manipulation of large, heavy objects demands forceful interaction across the entire body. However, such payloads shift a humanoid's center of mass and impose sustained loads across the upper body, challenging balance and command tracking. We present HULK, a whole-body control framework for forceful loco-manipulation. Using model predictive control (MPC) to guide reinforcement learning with predictions of the loaded dynamics, we train two teachers: one tracks arm motions under wrist forces, and the other locomotes while holding large objects against the body. A capture-point control barrier function augments the wrist-force teacher during training to improve balance under load. We distill both teachers into a single policy. Evaluation spans simulation and the Unitree G1. In simulation, the teacher with the barrier function achieves the lowest forward and lateral velocity tracking errors at 10 kg per arm among evaluated controllers and reduces aggregate divergent component of motion (DCM) excursion magnitude by 35.7% relative to MPC-guided reinforcement learning alone. Our wrist-force teacher withstands torso push disturbances of up to 130 N.

0.24

ImpedanceMimic: Learned Impedance as Action Space for Force-Accurate Motion Imitation

cs.RO cs.GRcs.LG ·2026-11-14 ·Matthias Jammot, Pembe Gizem Özdil
code/data similarity 0.67
Abstract · 39 pages, 13 figures, 28 tables. Project page: https://impedancemimic.github.io/ImpedanceMimic/

Recovering the forces behind a motion is ill-posed: different controllers can reproduce the same movement while generating different contact forces. Physics-based motion imitation resolves this ambiguity through simulation, but relies on fixed-gain proportional-derivative (PD) control. We show that fixed gains impose a trade-off between motion and force tracking, as contact and swing place conflicting demands on joint stiffness and damping. We introduce ImpedanceMimic, a reinforcement-learning policy that modulates per-joint stiffness, damping, and position setpoints from motion and force measurements, but requires only motion at inference. Across 129 participants and six activities, ImpedanceMimic outperforms matched fixed-gain PD under every integration scheme, direct-torque, muscle-actuated and existing imitation controllers in kinematic and force fidelity, approaching a dedicated motion-to-force regressor. By expressing gains as multiples of inertia-scaled priors, a single policy controls subject-specific biomechanical models without explicit anthropometric input and transfers zero-shot to unseen humans and a humanoid robot. The learned impedance forms a low-dimensional representation of motor state, shared across bodies, that generalizes to withheld activities: stiffness scales with load, and damping anticipates unloading.

0.17

Driving the Style: Velocity-Conditioned Whole-Body Control from Motion Demonstrations

cs.RO ·2026-11-16 ·Murad Dawood, Oezge Drama Berkenkamp, Antonio Gonzales Marin, Nutan Chen
similarity 0.67
Abstract

Achieving human-like behavior on humanoid robots has advanced rapidly by leveraging motion-capture demonstrations rather than hand-crafted cost functions. Adversarial imitation supplies a style signal that lets a policy fulfill a task while adhering to a desired style, but is difficult to transfer to hardware. Whole-body tracking through reinforcement learning instead deploys reliably and imitates demonstrations faithfully, but the resulting policies are conditioned on the future reference, so they require a reference trajectory at runtime and can fail when the motion differs from the training data. We train a humanoid policy directly with whole-body tracking rewards while restricting its actor to compact locomotion commands and robot observations. Reference motion supervises the reward and privileged critic, without supplying reference joint trajectories to the actor or requiring teacher distillation. The demonstrated style is thereby encoded in the policy through the tracking reward, so the robot reproduces the whole-body style under commanded velocities. Trained on a single random-walk motion and commanded with velocity profiles from held-out motions, our policy reaches at least 98\% success in simulation and completes every trial on the Unitree G1. On held-out slow walking, whole-body tracking drops to 2\% success and command-conditioned distillation to 56\%, and both fail on the real robot. On the trained motions, the joint position error of our policy stays within 8\% of whole-body tracking, and it has a higher style coverage than adversarial motion priors (AMP), which fell on the real robot without a safety crane.

0.20