How thermal cameras can help in ADAS vision systems

The vision component in an advanced driver-assistance system (ADAS) serves various purposes, from detecting various objects on the road, to recognizing speed limits and traffic signals. While visible cameras are mostly used in ADAS vision systems, they can be complemented by thermal solutions that can recognize and detect better in dark or inclement conditions.
 
Increasingly, the term ADAS has become more widespread amid efforts to make driving more autonomous. While there are various sensors employed in an ADAS system, supported with cutting-edge software, vision plays an important role, serving as the “eye” of the vehicle for the purposes or traffic sign recognition, lane departure warning, pedestrian/animal detection, parking assistance and collision avoidance, among others.
 
Against this backdrop, the ADAS vision system market is expected to see growth. According to a report by Research and Markets, the global camera ADAS market was worth US$2.7 billion in 2016. With the advent of semi-autonomous SAE level 3, and close-to fully autonomous level 4 vehicles, this is set to grow to an estimated $7.8 billion by 2021. 

SAE levels refer to the state of automation in a vehicle as designated by SAE International. There are six levels, from level 0 where the vehicle has no automation whatsoever, to level 5 where the car is fully autonomous and manual control is completely eliminated.

 

Visible vs. thermal

 
Visible cameras are already deployed today to add to a vehicle’s sense of vision. “Visible cameras, sonar and radar are already in use on production vehicles today at SAE automation level 2. SAE automation levels 3 and 4 test platforms have added light detection and ranging (LIDAR) to their sensor suite,” said Chris Posch, Director of Engineering for Automotive at FLIR Systems.
  
However, visible cameras alone have their drawbacks and this is where thermal cameras can come in handy, Posch said. “Tragically, as shown in recent Uber and Tesla accidents, the current sensors in SAE level 2 and 3 do not adequately detect cars or pedestrians.  Thermal cameras complement visible cameras and provide video that can be matched to image libraries to identify and classify objects in a scene and improve overall sensor suite performance,” he said.
 
According to Posch, thermal camera benefits are manifold. “Thermal, or longwave infrared (LWIR), cameras can detect and classify pedestrians in darkness, delivering improved situational awareness that results in more robust, reliable, and safe ADAS and AV,” he said.
 
“The Governors Highway Safety Association states the number of pedestrian fatalities in the U.S. has grown substantially faster than all other traffic deaths in recent years. They now account for a larger proportion of traffic fatalities than they have in the past 33 years. Pedestrians are especially at risk after dark when 75 percent of the 5,987 U.S. pedestrian fatalities occurred in 2016,” he added. “Thermal cameras can detect and classify pedestrians and other objects in darkness where they can see four times farther than headlights illuminate. They can see through most fog conditions and are unaffected by sun glare allowing them to help fill the gaps in the current ADAS sensor suite. “
 
FLIR, long recognized for offering cutting-edge thermal solutions for security applications, has also made progress in the automotive arena. “The industry standard for visible cameras is eleven items and FLIR currently offers a free starter dataset with over 14,000 annotated thermal images for training with five items including people, dogs, cars, other vehicles and bicycles.  A full set of eleven classes will be available in upcoming datasets from FLIR,” Posch said.