OpenAI said that they are using a technique called evolution strategy which is much faster than reinforcement learning.