A convex MPC locomotion controller and an RL policy for the Unitree G1 humanoid, built from scratch in MuJoCo. I run classical control and learned locomotion on the same platform.
BHEEMA is a model predictive control based locomotion system for the Unitree G1 humanoid. I built the whole control stack myself, following the MIT Cheetah 3 convex MPC framework and adapting it for a bipedal platform with passive ankles.
The G1 has no actuated ankles, and this matters a lot. Most bipedal MPC formulations assume 12 degrees of force control per step. The G1 only gives you 6, because each foot is basically a point contact. So I had to build the whole MPC around that constraint.
The system runs in MuJoCo with hardware matched actuator limits. The MPC plans at 16 Hz, the whole body controller runs at 200 Hz, and the physics runs at 2000 Hz. The robot stands as long as I want and walks forward with an alternating gait for over 20 seconds.
The MPC solves a convex QP over a finite horizon using single rigid body dynamics. It outputs contact forces for each stance leg. The whole body controller maps those forces to joint torques through Jacobian transposes, then adds gravity compensation and joint level PD feedback.
The swing leg controller uses 6D operational space impedance control to track foot trajectories from a Raibert heuristic footstep planner. The gait scheduler decides which legs are in stance and which are in swing at each moment.
The centroidal MPC models the G1 as a single rigid body with external contact forces at the feet. I linearize the rotation dynamics around the current orientation to keep the optimization convex, so the QP solves fast and reliably every cycle.
I approximate the friction cone constraints as pyramids and enforce them directly in the QP. The cost function penalizes deviations from the desired body pose and velocity, and I weight pitch and roll highest to keep the linearization valid.
The MPC outputs desired contact forces. The whole body controller turns those into motor commands that keep the robot balanced. Here is how it works at 200 Hz.
Next to the classical MPC stack, I trained an RL locomotion policy in MjLab as a second approach to bipedal walking. The policy learns straight from reward signals like velocity tracking, energy efficiency, gait quality, and posture stability, instead of from an explicit dynamics model.
The environment forks MjLab's well tested G1 velocity task and I tune the reward weights to shape the walking behavior I wanted. The observations, actions, terminations, and domain randomization come from the base config. The reward function is what makes the policy different.
I deploy the trained checkpoint through a pure NumPy inference bridge that handles the joint mapping between the 29 joint training model and the 43 joint MJCF, which includes BrainCo hands. The legs run PD torque control and the upper body runs position targets, which matches the actuator types in the physical hardware.
The reward function is where the locomotion knowledge lives. Each term already exists in MjLab's base config, so I tuned the weights to favor stable, efficient bipedal walking over general purpose locomotion.
I trained the policy on 29 joints (no hands). The deployment MJCF has 43 joints, which is 29 body joints plus 14 BrainCo hand joints interleaved through the kinematic tree.
The RL bridge handles this by building explicit qpos/qvel index maps at startup. For each of the 29 policy joints, it looks up the real address in the full MJCF instead of assuming naive sequential slicing. So the same bridge works no matter how many extra joints your MJCF has.
I split control by actuator type. The first 12 joints (legs) use motor actuators, so the bridge computes PD torques directly at 1000 Hz. The remaining 17 joints (waist, arms, wrists) use MuJoCo's built in position actuators and get joint angle targets directly.