In recent years, the popularization of AI technology has accelerated the application of automated systems in the real world. Among them, autonomous driving is the most typical one. Almost all car manufacturers are accelerating their R&D forces in this field.
However, the increasingly complex AI technologies and algorithms put forward higher requirements for the testing of automated systems, especially in safety testing/evaluation. Because self-driving cars have not yet been deployed on a large scale and natural driving data is scarce, many classical evaluation methods appear to be incompetent. Therefore, academia and the industry are looking for more effective testing methods.
Zhao Ding, assistant professor at Carnegie Mellon University, is one of them. He joined Carnegie Mellon University in 2018 and is currently the director of the Safe AI Lab. His research focuses on the safety of AI and explores how to assess the risks of automated vehicles quickly and effectively.
In 2016, Zhao published his dissertation "Accelerated Evaluation of Automated Vehicles," which was considered the first evaluation method to test automated vehicles (AVs) with statistical rigor.
The "Accelerated Evaluation" is meant to confirm, with high confidence, that a particular automated vehicle is safer than an average human-driving car. Because self-driving cars are not yet popular, and it is an even rarer case that they cause accidents, even using Monte Carlo simulation takes a long time and lacks statistical significance.
Zhao's method could accelerate the evaluation as quickly as 10,000 times depending on the scenarios. The core concept of his work is skewing the stochasticity of the environments to "artificially" increase the likelihood of accidents and mathematically "re-skewing" back the biased part of the estimator in the end.
Taking cut-in scenario as the example, if we can collect a large number of lane-change behavior from human drivers, build the stochastic distribution of the behavior, then the probability distribution functions can be manipulated to emphasize the "important" or "risky" behaviors and de-emphasize the "normal" or "boring" cases, achieving accelerated evaluation.
In Zhao's postdoc work, "Traffic Primitive," he presented the first unsupervised way to use machine-learning to automatically extract driving scenarios from "Safety Pilot," the world's biggest transportation dataset at that time.
Zhao's research work at MCity, the world's first testing center for the intelligent connected vehicles, resulted in a series of evaluation methods, which were included in the MCity's first white paper and became a part of the industry's testing protocol.
Zhao's Safe AI Lab has been collaborating with world-renowned self-driving companies including Ford, Uber, Toyota, Shanghai Motor, and Rolls Royce to make fundamental breakthroughs on AI safety and meet urgent societal needs.
In the future, in addition to continuing to promote related researches, Zhao also hopes to foster more AI safety talents and bridge the academia and the industry.
