Thermal sensors for ADAS and AEB systems, night vision and self-driving vehicles

1.1. The rapid increase in providing ADAS features as standard

Automobile manufacturers are now mainstreaming the use of ADAS systems in their vehicles. There are several reasons to explain why they are scaling up their efforts in this area, including constant changes to international regulations, the determination to improve road safety, and growing expectations from consumers. Modern vehicles now incorporate a wide range of features, such as automatic emergency braking (AEB), lane keep assist (LKA), adaptive cruise control (ACC), and vulnerable road user detection.

Night vision stands as one of the most critical challenges facing advanced driver-assistance systems.

With this in mind, thermal imaging delivers a tangible response that can work alongside other technologies. It allows ADAS systems to continue offering a high level of performance, even when visible light cameras, radars or lidar sensors are affected by light, the weather or lighting conditions.

1.2. Regulations: the impact of FMVSS 127 and Euro NCAP standards on AEB systems

In the United States, the new FMVSS 127 standard mandates that all new light vehicles must be fitted with an automatic emergency braking (AEB) system that is capable of detecting pedestrians in both daylight and darkness, and in normal driving conditions. This standard specifies strict requirements when it comes to night-time detection range and reliability.

A compliant AEB must demonstrate its ability to identify a pedestrian at a sufficient distance to avoid a collision, even in darkness or sun glare conditions. These tighter requirements have pushed the perception issue to the forefront and are prompting manufacturers to double down on their efforts to integrate such technologies as thermal imaging. 

In Europe, Euro NCAP performs a similar role by continually updating its active safety assessment protocols. Its test scenarios now comprise situations that are increasingly representative of real-life driving conditions, including the ability to detect pedestrians and cyclists in dark or low-light conditions. When awarding safety ratings, one of the key criteria is the actual performance of the AEB and vulnerable road user (VRU) protection systems.

1.3. Paving the way for autonomous driving (SAE Levels 3, 4 and 5)

The soaring rise in autonomous driving is significantly influencing how manufacturers design their embedded perception systems. 

Thermal imaging is playing an instrumental role in this change, since it delivers additional intel to the information provided by traditional sensors and helps enhance safety when visibility conditions are less than perfect.

The international classification issued by SAE International (Society of Automotive Engineers) defines six levels of driving automation (from 0 to 5). Levels 3, 4 and 5 correspond to the advanced forms of driving automation where the vehicle uses increasingly sophisticated thermal sensors to react and guarantee safety. Level 4 should be achieved by 2030, which reveals how fast developments are taking shape in this area.

Thermal imaging is not dependent on light, since it captures the infrared energy naturally emitted by objects, living beings and the environment. As such, it can provide stable, legible and exploitable perception information in the most challenging conditions.

2.1. AEB and pedestrian detection (PAEB): overcoming the night driving challenge

Most fatal accidents involving pedestrians happen at night or in poor visibility.

The tests performed with thermal sensors prove that they:

• Reliably detect pedestrians up to 100 meters depending on the resolution
• Offer resistance to headlight glare
• Ensure stable detection performance in urban and rural areas

To comply with the regulations, stopping distance calculations reveal that a system must recognize a pedestrian at no less than 46 meters when performing an emergency stop at 60 km/h.
Thermal sensors are capable of satisfying this requirement by a wide margin.

2.2. Embedded night vision: anticipating hazards

                                         Visible                                                                         Thermal 

Thermal cameras allow drivers — or autonomous systems — to see poorly-lit pedestrians or cyclists on the roadside, animals crossing the road, and non-reflective obstacles. Consequently, night vision can be construed as a native application of automotive thermal imaging.

2.3. Autonomous braking: the benefits with thermal imaging

There are two criteria that are usually hard to reconcile, i.e. reducing false positives and false negatives, but thermal imaging can improve both criteria at the same time. By lowering the number of cases where the braking system mistakenly triggers the brake pedal, it prevents phantom braking, while reinforcing the system's ability to detect actual hazardous situations.

Performance level measurements show that thermal imaging can significantly boost detection precision compared to a visible light camera alone, with an average precision increase of approximately 36%.                                                                                                          

3.1. SWaP: a key requirement

ADAS systems and autonomous platforms call for ultra-compact, lightweight and energy-efficient sensors. LYNRED's microbolometers specifically satisfy these requirements, since some models draw less than 1 W, which is a major advantage for incorporating these sensors into vehicles.

3.2. Field of vision, range and resolution

Studies show that pedestrians must have a height of approximately 20 pixels to be correctly classified by a neural network. In addition, the resolution and field of vision must be chosen in response to the intended use (city vs road).

LYNRED offers a wide choice of sensors that are available from the smallest format (QQVGA) to the largest format (SXGA), whole bringing the extended range needed for high-speed AEB scenarios and autonomous driving.

3.3. Embedded AI and signal processing

Integrated modules (such as advanced thermal imagers) are capable of:

• Incorporating automatic corrections
• Streamlining the camera development process
• Providing an image that is ready for AI inference

LYNRED offers one of the most extensive microbolometer ranges on the market, covering all ADAS and autonomous driving requirements.

4.1. Formats available: QVGA, VGA, XGA and SXGA

• QVGA: improved costs, entry-level AEB, short to medium-range detection
• VGA: benchmark for AEB systems, night vision and 360° perception
• SXGA: high resolution for long-range perception (autonomous vehicles)

4.2. 12 μm technology and new-generation 8.5 μm technology

Reducing the pixel pitch leads to:

• Increasingly compact optical systems
• Lower costs
• Easier integration

Tests show that the recognition range for the same lens aperture barely changes when the pixel pitch is reduced from 12 μm to 8.5 μm. This means that smaller sensors and optical systems can be used, which paves the way to thermal cameras with a lower price tag and without any trade-off in the performance required for ADAS systems.

4.3. Integrated modules: optimized images for AI

Modules featuring embedded algorithms guarantee stable image quality for neural networks, even in environments prone to changes in temperature.

5.1. A legacy of longstanding expertise in thermal imaging 

LYNRED has established a track record of 40 years in developing infrared detection solutions and can draw on its expertise across the entire IR spectrum, from SWIR to LWIR. Such maturity guarantees the technological stability and continuity that are so essential for the automotive industry with its lengthy development and production cycles.

5.2 Proven industrial capabilities for high-volume markets

LYNRED can harness its proven industrial capabilities to address the growing level of demand from manufacturers and OEMs. The company has already supplied several million detectors, as a testament to its complete expertise in high-volume production, reliable processes and performance repeatability.

LYNRED’s robust industrial processes improve supply chain security, and ensure conforming products and regular deliveries tailored to vehicle manufacturers’ production rates.

True to its determination to meet international requirements, LYNRED has already achieved ISO 9001, EN 9100 and ISO 14001 certification. As evidence of its commitment to excellence, LYNRED recently obtained the letter of conformance to IATF 16949. Specific to the automotive market, this standard requires companies to implement a quality management system that encourages continual improvement, prevents defects, and reduces nonconformities and waste across the supply chain.

5.3. Stable and predictable image quality for ADAS algorithms

The quality of the image delivered represents another decisive advantage. LYNRED’s sensors provide stable and uniform images, a low level of noise and embedded processing for improving contrast and legibility.

Such image quality is essential for guaranteeing reliable performance from the embedded algorithms and especially the neural networks that have been trained to detect and classify vulnerable road users. 

5.4. A partner focused on driving innovation and continual improvement

LYNRED firmly believes in continually investing in research and development to improve its detectors’ sensitivity, reduce their dimensions, rein in integration costs, and sense where the needs for autonomous systems will be heading in the future. This dynamic innovation policy provides manufacturers and OEMs with a gateway to scalable, futureproof technologies that are geared towards new mobility applications.

5.5. End-to-end support, from R&D through to mass production

Finally, LYNRED supports and guides its partners at each step of the product lifecycle, from the initial design stages through to environmental validation, final integration and production ramp-up.

LYNRED has developed the first open-source, large-scale thermal image dataset in Europe specifically for the automotive industry, advanced driver-assistance systems (ADAS) and other applications powered by artificial intelligence (AI). LYNRED Mobility Dataset is the solution for testing thermal imaging efficiency.

LYNRED Mobility Dataset is the only platform of its kind. It boasts over 250,000 thermal images and provides AI researchers with an unprecedented resource for evaluating the benefits that the thermal technology can bring to mobility applications, and much more besides.

LYNRED Mobility Dataset : Steréovision

Captured over the span of several years and seasons using a mix of thermal and visible-spectrum cameras, LYNRED Mobility Dataset offers a highly diverse and realistic set of road traffic scenarios. It is designed to help AI models learn how to detect obstacles and react to the full complexity inherent in today’s traffic environments - from pedestrians crossing snowy rural roads to vehicles navigating urban highways at night.

This dataset offers three complementary features:

• Multimodal detection: for training AI algorithms, with up to nine classes (of objects and people) in various weather conditions.
• Stereovision: for combining multimodal, stereo thermal IR and stereo visible-light RGB data, and tracking (video sequences), with perfectly synchronized images.
• Range estimation: for estimating the pedestrian detection range in various pedestrian automatic emergency braking conditions, which goes over and above the requirements in current regulations.