Using reinforcement machine learning to train a car
The car learnt to drive by itself in 15-20 minutes
Uses an algorithm called Deep Deterministic Policy Gradient (DDPG)
A UK company, Wayve, has designed their first ever self driving car which uses reinforcement machine learning. Self driving cars are one of the most popular and long awaited applications of artificial intelligence and machine learning. Until recently, we had seen only concepts or prototypes of these and obviously, sci-fi movies.
So far, the autonomous cars have included a ton of data including rules, several hours of leaning from that data and quite a lot of hardware. But now, Wayve, has designed an autonomous car that works on the concept of reinforcement learning. This helps the car to learn to drive in about 15-20 minutes!
How does this work?
That’s where it gets complicated. Researchers used a popular reinforcement learning algorithm called Deep Deterministic Policy Gradient (DDPG) which is a simple, yet, widely used algorithm in which the policy gradient can be estimated much more efficiently than its stochastic counterpart. This helped them to solve the problem of the car following the lane ahead of it. The architecture of the algorithm was a deep neural network consisting of 4 convolutional layers and 3 fully connected layers with just under 10,000 parameters compared to other state of the art image recognition algorithms consisting of more than a million parameters!
A single camera was used with the car to capture the surroundings and follow the road. All the processing was done in the car utilizing a single GPU( Graphics Processing Unit).
All the testing was carried out at Wayve’s workspace. A lot of testing was done in simulated environments to learn and understand and feed the hyperparameters into the reinforcement learning algorithm.
Every single time the car steered off track, it was manually steered on track by a human inside a car and rewarding it everytime it went wrong and corrected itself. It was reset after it travelled a certain distance. It took about 11 such training episodes to teach the car to follow a lane. The experiment was also carried in different weather conditions so that the car does not confuse itself if the road looked different.
Another company Drive.ai launched their very own self driving car in Texas, USA, along with Andrew Ng, you can read more about that here.
So Let’s summarise:
Maybe those days aren’t far off where we have unmanned cars driving people around and maybe we could get these cars to interact with each other and avoid traffic jams on the whole! What a great world that would be to live in!