Tesla’s progress with artificial intelligence and neural nets has propelled its Autopilot and Full Self Driving solutions to the front of the pack. This is the result of the brilliant work of a large team of Autopilot directors and staff, including Tesla’s Senior Director of AI, Andrej Karpathy. Karpathy presented Tesla’s methods for training its AI at the Scaled ML Conference in February. Along the way, he shared specific insights into Tesla’s methods for achieving the accuracy of traditional laser-based lidar with just a handful of cameras.

The secret sauce in Tesla’s ever-evolving solution is not the cameras themselves, but rather the advanced processing and neural nets they have built to make sense of the wide range and quality of inputs. One new technique Tesla’s AI team has built is called pseudo-lidar. It blends the lines between traditional computer vision and the powerful point map world of lidar.

Traditional lidar-based systems rely on an array of lidar hardware to provide an unparalleled view of the world around the vehicle. These systems leverage invisible lasers or similar tech to send a massive number of pings out into the world to detect surrounding objects.