Mobility engineers often note that autonomous vehicles are only as good as their sensors and the algorithms within. But the reliability and driving precision of an AV involves an equally important factor: the cleanliness of the sensor head, particularly that of lidar.
To understand what a lidar must endure, ride a motorcycle. A summertime tour on a road bike, or a day out on the trail coats the rider with dust, dirt, water and bug splatter. Aside from winter-time salt spray, all the muck and grit that sticks to a helmet visor and a rider’s jacket will also “blind” a lidar sensor of a car or truck – and jeopardize vehicle safety.
Unlike cameras, lidars are not located behind the windshield, which has IR-absorbent coatings. They must be mounted on an exterior surface without obstructions. Anything that obscures them – even water droplets that cling to the sensor head – can damage their optical stability. The Audi A8, an early application of lidar in a production ADAS array, has gaps in its grill where the lidar sensors are mounted for this reason.
“Keeping lidars clean is crucial to the operation of an AV,” Jason Fischer, chief engineer at Cruise Automation, told SAE International during a media event in early 2020. “It’s a technical challenge that has the industry engaged in finding solutions.”
Ford’s AV development team created a buzz last year with its approach to understanding methods for deflecting insects before they hit test vehicles’ sensor arrays. They interviewed zoologist and University of Florida professor Mark Hostetler, author of the book That Gunk on Your Car (and who also created a phone app that can identify various bug splatter) for his advice. The Ford team even built a laboratory contraption that fires insects at high velocity into sensor heads, to better understand how to clean them.
Finally, they built a set of passive-aerodynamic shields for installation on the roofs of Ford’s self-driving prototype fleet. Nicknamed “tiara” due to their crown-like shape, the devices create air currents that deflect insects from the rooftop sensors. The Ford AV team has applied for many patents related to self-driving cleaning technologies.
Valeo and Continental are among industry suppliers that have developed fully automatic cleaning systems for AV lidars and cameras. The technologies typically use retractable liquid-jet nozzles that spray a precise amount of cleaning fluid (stored in an onboard reservoir) onto the sensor lens or face. Some include drying and de-icing heaters. Valeo’s Centricam system cleans by means of centrifugation.
Ultrasonic benefits and challenges
Then there is ultrasonic cleaning, a process widely used in industries including electronics, medical (kidney stones), jewelry, firearms and automotive fuel injection. Digital cameras have featured ultrasonics to remove dust and dirt from internal surfaces since Olympus introduced the feature in 2003. Ultrasonic cleaning works well on sensors because of their relatively small area to clean. The technology employs piezoelectronics (PZT) to generate an ultrasound frequency of around 35 kHz. Rather than pressure-washing the dirt with a liquid jet, a thin film of water or cleaning fluid is applied. The vibration of the lens transfers the dirt into the fluid. It then atomizes the fluid from the surface, taking the dirt with it. The ultrasonic actuation can do the following:
- Prevent weakly-bound dust and dirt from sticking
- Use the high acceleration of the surface to cause hard particles to ‘bounce’ rather than ‘stick’
- Reduce the friction coefficient of the surface by forming an air cushion which prevents particles from sticking
- Atomize and eject liquids from the surface
Atomization of liquid requires the surface to move fast enough to break the surface tension. It is the most effective mechanism for removing surface crud from a sensor, according to Tom Jellicoe, an opto-electronics expert and leader of the autonomous driving and mobility practice at TTP, a technology consultancy in Cambridge, England.
“It’s amazing how clean the sensor surfaces have to be, particularly when you push the limits of [lidar] range,” he said. “The electronics are searching for the tiniest of signals. Anything that obscures it quite damaging; the sensor acuity can really be undone by the slightest imperfection on the front surface.”
The finite-element simulations conducted by TTP of a typical lens geometry suggests that to clean such a small area ultrasonically “requires about the same level of electronic actuation as a standard doorbell buzzer,” Jellicoe– encouraging results from a power-consumption and robustness perspective. He added that the results are based on “a relatively simple model” and additional challenges remain.
A challenge with using ultrasonic PZT actuators is in determining how hard the actuator needs to be driven for effective cleaning – and its effect on the vehicle battery’s state of charge as well as vibration and harshness.
TTP has deep experience in ultrasonic technology, having developed cleaning solutions for semiconductor wafer fabs, industrial printer heads, food prep and even a system that works inside a vacuum cleaner. As AV development has proliferated, OEMs and suppliers are focused on ensuring all vehicle sensors retain optimum functionality in all driving conditions.
“We’ve built up computational models for exactly how we’d design a system for different geometries and materials,” Jellicoe explained. “Typically, how this works is we understand the theory and we produce computational models for what the system should look like. We then prototype them in our lab according to our client’s specification, then we test and characterize them in our metrology suite.”
Integrated with the sensor
The sensor-cleaning topic is essentially cost motivated, Jellicoe said. “We started looking at it about four years ago when a Michigan-based Tier 1 asked us to look at ways they could integrate cleaning into their automotive cameras,” he recalled. “We modeled a few concepts and settled on ultrasonic cleaning as most effective.”
More recently, BMW engineers engaged Jellicoe’s team about ways to increase the stability of their AV algorithms by keeping sensors clean. The TTP engineers pointed out many issues related to using a liquid-jet wiper system – difficult to accommodate the many different sensor geometries; need for a large fluid tank on the vehicle, and addition of multiple wiper systems.
“There’s also the integration challenge,” Jellicoe noted. “Every time an OEM adds or moves a sensor, a Tier 1 has to reconfigure their liquid-jet cleaning system to optimize the geometry and the spray. So, the OEMs are keen to reduce and maybe eliminate the volume of liquid required.”
Liquid-jet wiper-based cleaning systems also require large-capacity fluid reservoirs, prompting Valeo North American CTO Jim Schwyn to quip: “Are we going to take the gasoline tank from a car and replace it with a windshield-washer reservoir to keep things clean?”
Beyond the threat of being knocked out by flying insects, dirt, and road debris, a significant concern for lidar surfaces is condensation from fog and road spray that can prove as difficult to remove quickly as solid material, Jellicoe said.
Most AVs in development now have their sensor arrays in the open; in the future they will be integrated into the bodywork – driving changes in how ultrasonic cleaning technology is applied.
“The way we think it will work best – and certainly what the OEMs and AV companies we speak with are really excited about – is integration of the cleaner system into the sensor itself,” Jellicoe said. “Looking at the proliferation of sensors on the outside of vehicles, Zoox [the AV company acquired by Amazon] has 18 cameras and 8 lidars on their vehicles. So it’s not feasible to have nearly 30 liquid jets located around the vehicle and draining the liquid tank every hour. They need something ‘smarter’ than that.”
Equally, no OEM that has multiple sensors on a Level-4 vehicle will want a bespoke cleaning solution, Jellicoe asserted. “The OEM is not going to deploy additional products in addition to the sensors to keep them clean. Cost is going to be ‘king’ in lidar – it’s something we hear from our clients every day,” he said. “So, there is a lot of opportunity for the sensor manufacturers to integrate the cleaning system into their products.”
Sensors located on the vehicle exterior have, in most cases, a heating element behind the front fascia for de-icing. Jellicoe noted that ultrasonic cleaning “can effectively replace the heater,” using the ultrasonics to de-ice. “And it uses far less power. So, you’re not asking for extra space to add the ultrasonics; you’re just adding a more-clever technology.” When it is mentioned that this all could be described as “TTP Inside” (with apologies to Intel), Jellicoe chuckled. “I’ve actually heard it described like that,” he said.Continue reading »