TINYBOT FORGE
Home › Learn › Make a robot teach itself to walk in Python

Make a robot teach itself to walk in Python

PythonMuJoCoCMA-ESNo GPUUpdated September 30, 2026

A four-legged robot starts out falling over. One minute of simulated evolution later, it gallops. Here is all the code, tested, with the output we got.

Tested on 2026-09-30 with Python 3, MuJoCo 3.x and cma 4.x. The output shown below is from our own run.

You don't need a GPU, a neural network or a robotics degree for this. A four-legged robot, a physics engine and an evolution strategy fit in 45 lines of Python, and on an ordinary laptop the robot goes from falling over to galloping in about a minute. Copy the code below, run it, and watch it happen.

What you need

MuJoCo is a free, open-source physics engine (the same one used in a lot of robotics research). cma is a Python implementation of CMA-ES, an evolution strategy that's very good at tuning a few dozen numbers when you can only measure "how well did it do".

The idea in four steps

  1. Body. A box torso with four legs. Each leg has a hip and a knee, and each joint has a motor that moves to whatever angle we ask for.
  2. Brain. Every motor follows a sine wave: angle = amplitude · sin(2π · f · t + phase) + offset. Eight motors × 3 numbers + 1 shared frequency = 25 numbers. Those are the robot's genes.
  3. Test. Drop the robot on the floor, run the physics for 5 seconds, measure how far forward it got. Fell on its back? Subtract a metre.
  4. Evolve. CMA-ES creates 24 variations of the genes, tests all of them, and shifts toward the ones that went farthest. Repeat 100 times.

Nobody writes the gait. It gets discovered, which is why evolved robots often move in ways a human wouldn't design.

The code: walker.py

# walker.py - a 4-legged robot teaches itself to walk (MuJoCo + CMA-ES)
# pip install mujoco cma numpy      then: python walker.py
import numpy as np, mujoco, cma

LEG = """<body pos="{x} {y} 0"><joint name="hip{i}" type="hinge" axis="0 1 0" range="-60 60"/>
  <geom type="capsule" fromto="0 0 0 0 0 -0.12" size="0.02"/>
  <body pos="0 0 -0.12"><joint name="knee{i}" type="hinge" axis="0 1 0" range="-70 70"/>
    <geom type="capsule" fromto="0 0 0 0 0 -0.12" size="0.018"/></body></body>"""
CORNERS = [(0.15, 0.1), (0.15, -0.1), (-0.15, 0.1), (-0.15, -0.1)]
XML = f"""<mujoco><option timestep="0.004"/>
<default><joint damping="0.5" armature="0.01"/><geom friction="1 0.005 0.0001" density="1000"/>
  <position kp="8" ctrlrange="-1.2 1.2" ctrllimited="true"/></default>
<worldbody><geom type="plane" size="20 20 0.1"/>
 <body name="torso" pos="0 0 0.3"><freejoint/><geom type="box" size="0.18 0.12 0.03" density="300"/>
  {''.join(LEG.format(x=x, y=y, i=i) for i, (x, y) in enumerate(CORNERS))}
 </body></worldbody>
<actuator>{''.join(f'<position joint="hip{i}"/><position joint="knee{i}"/>' for i in range(4))}</actuator>
</mujoco>"""

model = mujoco.MjModel.from_xml_string(XML)
N = model.nu                      # 8 motors
SECONDS = 5

def run(genes, seconds=SECONDS):
    """Genes -> distance walked. Every motor follows amp*sin(2*pi*f*t + phase) + offset."""
    amp, phase, offset = genes[:N], genes[N:2*N], genes[2*N:3*N]
    freq = np.clip(1.5 + genes[-1], 0.5, 3.0)   # one shared frequency, 0.5-3 Hz
    d = mujoco.MjData(model)
    while d.time < seconds:
        d.ctrl[:] = amp * np.sin(2 * np.pi * freq * d.time + phase) + offset
        mujoco.mj_step(model, d)
    upright = d.xmat[1][8] > 0    # torso z-axis still pointing up?
    return d.qpos[0] if upright else d.qpos[0] - 1.0

if __name__ == "__main__":
    es = cma.CMAEvolutionStrategy(np.zeros(3 * N + 1), 0.5, {"popsize": 24, "seed": 1, "verbose": -9})
    for gen in range(1, 101):
        population = es.ask()
        distances = [run(g) for g in population]
        es.tell(population, [-x for x in distances])   # CMA-ES minimises, so negate
        if gen % 10 == 0:
            print(f"gen {gen:3d}   best {max(distances):5.2f} m   best ever {-es.result.fbest:5.2f} m")

    np.save("champion.npy", es.result.xbest)
    print("saved champion.npy  ->  python watch.py")
Download walker.py

Run it

$ python walker.py
gen  10   best  0.36 m   best ever  0.95 m
gen  20   best  1.84 m   best ever  1.84 m
gen  50   best  3.85 m   best ever  3.85 m
gen  80   best  5.16 m   best ever  5.16 m
gen 100   best  4.96 m   best ever  5.16 m
saved champion.npy  ->  python watch.py

That's our actual output (MuJoCo 3.x, seed 1, about 60 seconds on one CPU core). The robot is 36 cm long and ends up covering about 5 metres in 5 seconds. Your numbers will differ with another seed or MuJoCo version, and that's part of the fun: different runs discover different gaits.

Watch your robot: watch.py

# watch.py - replay the champion in a 3D window  (macOS: run with mjpython)
import time, numpy as np, mujoco, mujoco.viewer
from walker import model, N          # reuses the robot (guarded below)

genes = np.load("champion.npy")
amp, phase, offset = genes[:N], genes[N:2*N], genes[2*N:3*N]
freq = np.clip(1.5 + genes[-1], 0.5, 3.0)
d = mujoco.MjData(model)
with mujoco.viewer.launch_passive(model, d) as v:
    while v.is_running():
        d.ctrl[:] = amp * np.sin(2 * np.pi * freq * d.time + phase) + offset
        mujoco.mj_step(model, d)
        v.sync()
        time.sleep(model.opt.timestep)
Download watch.py

A 3D window opens with your champion walking. Drag to rotate, scroll to zoom. On macOS start it with mjpython watch.py (installed with the mujoco package).

Want 2, 4, 6 and 8 legs, stairs, ice and evolving bodies?

This page is the minimal version. Robot Evolution Lab is the full code behind our videos: every leg count, stairs and ice terrain, body-shape evolution, a video recorder, robot races and pre-evolved champions. US$7.

See what's inside

Why the early generations look useless

For the first 10–30 generations most robots twitch, tip over or go backwards. That's normal: from all-zero genes there's no gait to improve yet. CMA-ES also adapts its own step size, so once it finds a direction that helps, progress speeds up. You can see that jump in the output above between generation 40 and 50.

Five things to try next

  1. Change the seed ("seed": 2). A different seed often finds a completely different gait: a bound, a trot, a shuffle.
  2. Reward something else. Return the torso's maximum height instead of distance and you'll evolve jumpers.
  3. Punish energy. Subtract 0.001 * np.sum(d.ctrl**2) every step and the gaits get smoother and calmer.
  4. Make the floor slippery. Change friction="1 ..." to friction="0.1 ...". Gaits evolved on normal ground usually fail on ice, which is what we showed in this video.
  5. More legs. Add corners to CORNERS. Six and eight legs change everything; in our race eight legs won on flat ground.

FAQ

Is this reinforcement learning?

No. There's no neural network and no gradient. It's black-box optimisation: CMA-ES only sees "these genes walked 3.2 m". Reinforcement learning (PPO, SAC) can learn richer controllers that react to sensors, but it's far slower to set up and train. Evolving an open-loop sine-wave gait is the fastest way to see a robot learn to walk.

Why a sine wave?

Walking is rhythmic, so a repeating signal per joint is a natural starting point. Biology does something similar with central pattern generators, the circuits in the spinal cord that produce rhythmic motion.

Can it run on a real robot?

The same idea works on a small servo robot, but a gait evolved in simulation rarely transfers perfectly. Real motors, friction and weight differ from the model, a problem known as the sim-to-real gap.

Are your YouTube videos made with this?

Yes, the same method with more options and a renderer. They're physics simulations, not AI-generated video. The full version is Robot Evolution Lab.

Get Robot Evolution Lab · US$7

New robot experiments every day

Stairs, ice, evolving bodies, robot vacuums: all real physics simulations.

Subscribe on YouTube

CMA-ES or reinforcement learning?

We gave this exact robot to PPO too, with the same test and the same number of trials. See which one walked farther, and which one cheated.

See the experiment