Deep Evolutionary Reinforcement Learning creates virtual agents that can evolve both in their physical structure and learning capacities.