Motivations, Opportunities and Challenges
Dr. N. Kemal Ure – Director of AI, Eatron Technologies

Part 1: Introduction

It is well-embraced that Artificial Intelligence (AI) continues to be one of the most impactful technologies of our times. AI has completely transformed how we approach engineering problems; replacing rule and heuristic-driven design approaches with data-driven algorithms. It is also evident that AI is now an essential part of autonomous systems, whether it is self-driving car, service robotics or drones. Most, if not all perception systems in autonomous driving systems have some form of AI-based model in their operation. Deep learning, the well-acclaimed hero of modern AI, enabled our self-driving cars to gain an unprecedented degree of self-awareness. We can now detect pedestrians, segment drivable areas and predict the trajectories of surrounding traffic with higher accuracy under variety of different conditions. That being said, an integral part of the puzzle is still missing. At the end of the day, sensing/perception is only half of the problem; now that we can fuse all this information, what do we do with it? In other words, how should our autonomous car optimize its decisions to execute actions that leads to superior performance, while ensuring safety? That is where the reinforcement learning (RL) steps in.

Simply put, RL is the machine learning for control. That is, instead of explicably designing control laws or decision policies, we let data collected from the environment dictate what should the optimal mapping from sensing to control should be. The main premise of RL is, it can be applied to almost any decision- making problem, and the algorithm can learn purely from data collected with interactions form the environment, without any prior knowledge about the problem dynamics. RL started to gain massive popularity in recent years, with successful demonstrations in games, robotics, recommendation systems, and automated trading. In the context of autonomous driving, RL has many different applications, ranging from optimizing high level decisions such as lane changing to act as a supervisor for switching between different low level control laws.

Although RL already started to make significant impact on autonomous driving, there are still many open areas, opportunities to pursue, and challenges to tackle. Specifically, although there is a vast literature on doing RL in simulations or small-scale testbeds, demonstrating the potential of RL in real world autonomous driving is still an understudied area. The main objective of these series of blog posts is to highlight the motivations and opportunities in applying RL to real world autonomous driving, as well as emphasizing some unique scientific and engineering challenges that needed to be solved in near future. In the next blog post, we will go through the motivations and opportunities for applying RL to real world autonomous driving.