John Schulman

John Schulman, a research scientist at OpenAI, has created some of the key algorithms in a branch of machine learning called reinforcement learning. It’s just what it sounds like: you train AI agents in the same way you might train a dog, by offering a treat for a correct response. For a machine, the “treat” might be to rack up a high score in a video game.

Which explains why Schulman is so excited about the 1991 video game Sonic the Hedgehog. The game, he says, is a perfect benchmark for testing how well new machine-­learning algorithms transfer learned skills to new situations. Since Sonic is the world’s fastest hedgehog, the game moves rapidly, and it also depicts some interesting physics. Once an AI agent learns how to play, it’s easy for researchers to test its ability to transfer that knowledge to different scenarios.

These algorithms, once trained, might be applied in the real world, and they can be used to improve robot locomotion. Traditional approaches have been specialized for certain situations—which means that on new terrain, a robot programmed using older methods might fall down. One that uses reinforcement learning, ­Schulman hopes, would be able to get back up and try new things until it solves the problem.

—Katherine Bourzac