Make a robot teach itself to walk in Python
A four-legged robot starts out falling over. One minute of simulated evolution later, it gallops. Here is all the code, tested, with the output we got.
You don't need a GPU, a neural network or a robotics degree for this. A four-legged robot, a physics engine and an evolution strategy fit in 45 lines of Python, and on an ordinary laptop the robot goes from falling over to galloping in about a minute. Copy the code below, run it, and watch it happen.
What you need
- Python 3.9 or newer (Windows, macOS or Linux)
- Three packages:
pip install mujoco cma numpy
MuJoCo is a free, open-source physics engine (the same one used in a lot of robotics research). cma is a Python implementation of CMA-ES, an evolution strategy that's very good at tuning a few dozen numbers when you can only measure "how well did it do".
The idea in four steps
- Body. A box torso with four legs. Each leg has a hip and a knee, and each joint has a motor that moves to whatever angle we ask for.
- Brain. Every motor follows a sine wave:
angle = amplitude · sin(2π · f · t + phase) + offset. Eight motors × 3 numbers + 1 shared frequency = 25 numbers. Those are the robot's genes. - Test. Drop the robot on the floor, run the physics for 5 seconds, measure how far forward it got. Fell on its back? Subtract a metre.
- Evolve. CMA-ES creates 24 variations of the genes, tests all of them, and shifts toward the ones that went farthest. Repeat 100 times.
Nobody writes the gait. It gets discovered, which is why evolved robots often move in ways a human wouldn't design.
The code: walker.py
# walker.py - a 4-legged robot teaches itself to walk (MuJoCo + CMA-ES)
# pip install mujoco cma numpy then: python walker.py
import numpy as np, mujoco, cma
LEG = """<body pos="{x} {y} 0"><joint name="hip{i}" type="hinge" axis="0 1 0" range="-60 60"/>
<geom type="capsule" fromto="0 0 0 0 0 -0.12" size="0.02"/>
<body pos="0 0 -0.12"><joint name="knee{i}" type="hinge" axis="0 1 0" range="-70 70"/>
<geom type="capsule" fromto="0 0 0 0 0 -0.12" size="0.018"/></body></body>"""
CORNERS = [(0.15, 0.1), (0.15, -0.1), (-0.15, 0.1), (-0.15, -0.1)]
XML = f"""<mujoco><option timestep="0.004"/>
<default><joint damping="0.5" armature="0.01"/><geom friction="1 0.005 0.0001" density="1000"/>
<position kp="8" ctrlrange="-1.2 1.2" ctrllimited="true"/></default>
<worldbody><geom type="plane" size="20 20 0.1"/>
<body name="torso" pos="0 0 0.3"><freejoint/><geom type="box" size="0.18 0.12 0.03" density="300"/>
{''.join(LEG.format(x=x, y=y, i=i) for i, (x, y) in enumerate(CORNERS))}
</body></worldbody>
<actuator>{''.join(f'<position joint="hip{i}"/><position joint="knee{i}"/>' for i in range(4))}</actuator>
</mujoco>"""
model = mujoco.MjModel.from_xml_string(XML)
N = model.nu # 8 motors
SECONDS = 5
def run(genes, seconds=SECONDS):
"""Genes -> distance walked. Every motor follows amp*sin(2*pi*f*t + phase) + offset."""
amp, phase, offset = genes[:N], genes[N:2*N], genes[2*N:3*N]
freq = np.clip(1.5 + genes[-1], 0.5, 3.0) # one shared frequency, 0.5-3 Hz
d = mujoco.MjData(model)
while d.time < seconds:
d.ctrl[:] = amp * np.sin(2 * np.pi * freq * d.time + phase) + offset
mujoco.mj_step(model, d)
upright = d.xmat[1][8] > 0 # torso z-axis still pointing up?
return d.qpos[0] if upright else d.qpos[0] - 1.0
if __name__ == "__main__":
es = cma.CMAEvolutionStrategy(np.zeros(3 * N + 1), 0.5, {"popsize": 24, "seed": 1, "verbose": -9})
for gen in range(1, 101):
population = es.ask()
distances = [run(g) for g in population]
es.tell(population, [-x for x in distances]) # CMA-ES minimises, so negate
if gen % 10 == 0:
print(f"gen {gen:3d} best {max(distances):5.2f} m best ever {-es.result.fbest:5.2f} m")
np.save("champion.npy", es.result.xbest)
print("saved champion.npy -> python watch.py")
Run it
$ python walker.py gen 10 best 0.36 m best ever 0.95 m gen 20 best 1.84 m best ever 1.84 m gen 50 best 3.85 m best ever 3.85 m gen 80 best 5.16 m best ever 5.16 m gen 100 best 4.96 m best ever 5.16 m saved champion.npy -> python watch.py
That's our actual output (MuJoCo 3.x, seed 1, about 60 seconds on one CPU core). The robot is 36 cm long and ends up covering about 5 metres in 5 seconds. Your numbers will differ with another seed or MuJoCo version, and that's part of the fun: different runs discover different gaits.
Watch your robot: watch.py
# watch.py - replay the champion in a 3D window (macOS: run with mjpython)
import time, numpy as np, mujoco, mujoco.viewer
from walker import model, N # reuses the robot (guarded below)
genes = np.load("champion.npy")
amp, phase, offset = genes[:N], genes[N:2*N], genes[2*N:3*N]
freq = np.clip(1.5 + genes[-1], 0.5, 3.0)
d = mujoco.MjData(model)
with mujoco.viewer.launch_passive(model, d) as v:
while v.is_running():
d.ctrl[:] = amp * np.sin(2 * np.pi * freq * d.time + phase) + offset
mujoco.mj_step(model, d)
v.sync()
time.sleep(model.opt.timestep)
A 3D window opens with your champion walking. Drag to rotate, scroll to zoom. On macOS start it with mjpython watch.py (installed with the mujoco package).
Want 2, 4, 6 and 8 legs, stairs, ice and evolving bodies?
This page is the minimal version. Robot Evolution Lab is the full code behind our videos: every leg count, stairs and ice terrain, body-shape evolution, a video recorder, robot races and pre-evolved champions. US$7.
See what's insideWhy the early generations look useless
For the first 10–30 generations most robots twitch, tip over or go backwards. That's normal: from all-zero genes there's no gait to improve yet. CMA-ES also adapts its own step size, so once it finds a direction that helps, progress speeds up. You can see that jump in the output above between generation 40 and 50.
Five things to try next
- Change the seed (
"seed": 2). A different seed often finds a completely different gait: a bound, a trot, a shuffle. - Reward something else. Return the torso's maximum height instead of distance and you'll evolve jumpers.
- Punish energy. Subtract
0.001 * np.sum(d.ctrl**2)every step and the gaits get smoother and calmer. - Make the floor slippery. Change
friction="1 ..."tofriction="0.1 ...". Gaits evolved on normal ground usually fail on ice, which is what we showed in this video. - More legs. Add corners to
CORNERS. Six and eight legs change everything; in our race eight legs won on flat ground.
FAQ
Is this reinforcement learning?
No. There's no neural network and no gradient. It's black-box optimisation: CMA-ES only sees "these genes walked 3.2 m". Reinforcement learning (PPO, SAC) can learn richer controllers that react to sensors, but it's far slower to set up and train. Evolving an open-loop sine-wave gait is the fastest way to see a robot learn to walk.
Why a sine wave?
Walking is rhythmic, so a repeating signal per joint is a natural starting point. Biology does something similar with central pattern generators, the circuits in the spinal cord that produce rhythmic motion.
Can it run on a real robot?
The same idea works on a small servo robot, but a gait evolved in simulation rarely transfers perfectly. Real motors, friction and weight differ from the model, a problem known as the sim-to-real gap.
Are your YouTube videos made with this?
Yes, the same method with more options and a renderer. They're physics simulations, not AI-generated video. The full version is Robot Evolution Lab.
Get Robot Evolution Lab · US$7
New robot experiments every day
Stairs, ice, evolving bodies, robot vacuums: all real physics simulations.
Subscribe on YouTubeCMA-ES or reinforcement learning?
We gave this exact robot to PPO too, with the same test and the same number of trials. See which one walked farther, and which one cheated.
See the experiment