Self-driving cars get even better at driving in the dark

This article is our exclusive report IEEE Journal Watch Series In collaboration with IEEE Xplore.

A team of researchers in China has discovered a way to help self-driving cars “see better” in the dark — improving the cars’ ability to drive by more than 10 percent. The secret to the researchers’ success lies in a decades-old theory about how the human eye works.

One way self-driving cars navigate is by using an array of cameras, each equipped with special filters that can discern the polarization of incoming light. Polarization refers to the direction in which light waves oscillate as they travel — and it can provide a wealth of information about the object it last reflected off, including the object’s surface features and details.

However, while polarizing filters provide self-driving cars with additional information about surrounding objects, they also have some drawbacks.

“While this dual-filter design provides more information, it also makes it more difficult to capture photons at night,” said Yang Lu, a doctoral student at the University of Chinese Academy of Sciences. “As a result, the image quality of the polarization camera drops dramatically in low-light conditions, with details and clarity being more severely affected.”

To address this question, Lu and his colleagues employed a theory that attempts to explain why humans can discern colors relatively well in low-light conditions. The Retinex theory posits that our visual system is able to discern light in two different ways: its reflected and illuminated components. Importantly, even in low-light conditions, our eyes and brains are able to compensate for illumination changes in the light enough to discern colors.

Lu’s team applied this concept to their self-driving car navigation system, which processes the reflective and luminous properties of polarized light separately. One algorithm – trained using real data of the same images in light and dark conditions – compensates for changes in brightness, just like our own visual system. The second algorithm processes the reflective properties of the incoming light and removes background noise.

Traditional self-driving cars tend to only deal with the reflective properties of light, but this dual approach provides better results. “In the end, we got a [more] A clearly presented image,” Lu said.

In a study published on August 8 IEEE Transactions on Intelligent VehiclesThe researchers tested their new method, RPLENet.

First, the team used real data from dim environments for simulation to verify whether their method could produce better low-light imaging. Then, they installed a camera that relied on RPLENet on a car and tested it in a real night scene. The results showed that the new method can improve driving accuracy by about 10% in autonomous driving algorithm experiments.

Lu noted that the new method could potentially make self-driving cars safer. “We achieved excellent results in our tests, especially in real nighttime scenarios, proving that our method has potential for practical applications,” he said.

However, one challenge of the RPLENet approach is that it requires extensive training on datasets that are difficult to obtain (e.g., images of the same object under different lighting scenarios).

“In the future, we plan to further explore weakly supervised and unsupervised learning methods to reduce the reliance on large amounts of labeled data,” Lu said. “This will accelerate algorithm development and help provide more efficient and cost-effective solutions in real-world applications.”