How thermal imaging helps save lives while complying with the FMVSS127 regulation

The detection of pedestrians at night is a crucial issue for road safety. 88% of pedestrians move on poorly maintained roads, lacking sidewalks, safe crossings, or lighting, where vehicles often travel at speeds exceeding 60 km/h. This situation means that 10% of injured pedestrians lost their lives in 2023 (Sources: Research-Report-Pedestrian-Detection.pdf).

With regard to the situation, frequent changes have been done to international regulations and safety standards aiming to strengthen the protection of vulnerable road users, such as pedestrians, cyclists, and motorcyclists.

In 2029, a significant step forward will be taken with the setting to effect of regulation FMVSS 127, which tightens the standards for nighttime emergency braking systems for all passenger cars and trucks sold in the USA that weigh less than 4.5 tons.

In this context, how can thermal imaging enhance existing safety systems?

1. Retrospective of Key Safety Features in Automotive

Automotive safety has significantly evolved over the decades, with the introduction of numerous devices and technologies aimed at protecting vehicle users and reducing the risk of accidents. Discover a retrospective of the key safety features in automotive:

2. Presentation of FMVSS 127: Stakes and Opportunities

The FMVSS No. 127 (Federal Motor Vehicle Safety Standard No. 127) regulation from NHTSA (National Highway Traffic Safety Administration) is a legislative response to the concern causing statistics of road accidents.
It requires passenger vehicles and trucks in the United States to be equipped as standard with an AEB (Automatic Emergency Braking) system and a PAEB (Pedestrian Automatic Emergency Braking) system effective at night in all passenger cars and vans by 2029.
More specifically, it stipulates that vehicles must be equipped with an AEB system and a FCW (Forward Collision Warning) system operating at speeds between 10 km/h (6 mph) and 145 km/h (90 mph).

• The AEB system must have the capacity to prevent collisions with vehicles at speeds up to 100 km/h (62 mph) and detect pedestrians, both day and night, with low beam headlights and without urban lighting. Activation of automatic braking is required at speeds below 73 km/h (45.3 mph) if a pedestrian is detected.

• The FCW system must emit both audible and visual warnings to alert the driver of a collision risk at speeds of up to 145 km/h (90 mph).

Although AEB is no new technology—about 90% of new vehicles in the United States are already equipped—the FMVSS 127 standard aims to increase this rate to 100% by making the technology mandatory, while improving its performance and capabilities.

          2.1 - The Challenges of Excellence

The combination of high-speed testing and nighttime pedestrian detection, without urban lighting and using low beam headlights, may require more advanced sensor systems than those currently used, such as long-range radars and LiDAR:

• High-speed collision prevention: The FMVSS standard requires AEB and FCW tests at record speed: 70 km/h and 100 km/h respectively. As a consequence, braking distances become longer, requiring sensors with much larger field of view.

• Complexity with nighttime testing: Another major challenge lies in nighttime testing for PAEB. The FMVSS 127 regulation requires testing in complete darkness, and in the most difficult cases, with only low beam headlights to illuminate the scene in front of the vehicle. This makes pedestrian detection more challenging for sensor systems.

• Strict success requirements: The FMVSS 127 regulation requires a success rate of 100%, with no margin for error, unlike other international AEB standards that allow a certain failure rate.

          2.2 – Necessary Technological Progress

It is clear that FMVSS 127 is one of the most demanding regulations in terms of active safety.

In Europe, Euro NCAP (European New Car Assessment Programme) serves as benchmark for assessing the safety of best-selling cars in Europe, but test speeds are different: they do not exceed 80 km/h (50 mph), and collision mitigation is tolerated, therefore not requiring a 100% success rate. However, the AEB protocols of Euro NCAP are even more demanding as they cover a wider range of speeds and scenarios, including cyclists, motorcyclists, curves at intersections, winding roads, and lane changes, which require lateral sensors and a more sophisticated AEB system.

Euro NCAP will also introduce performance assessments in poor visibility conditions starting in 2026, such as the absence of urban lighting or pedestrians dressed in different colors. This approach, called the “Robustness layer,” will increase the need for robust detection.

The requirement for AEB systems to prove optimal functioning in all weather conditions, including at night, is pushing for an increase of the level of vehicle autonomy from current level 1 and 2 (and even 3 for some newer vehicles) to level 4 by 2030, meaning vehicles capable of operating without human intervention in controlled environments. This specific operational context is covered in the concept of ODD (Operational Design Domain), in which thermal imaging enables to expand context scope.

3. Thermal imaging response

          3.1 – Limitations in existing systems

Since 2022, all new European vehicles are equipped with active braking systems. These systems have normatively inbuilt cameras and radars for high-end vehicles. The diagram below presents the main sensors embedded.

Main sensors for AEB

In low-light conditions, the performance of the RGB camera and radar reaches its limits. Their ability to perceive the environment weakens due to a lack of light, weather conditions such as rain, fog, or snow, as well as external factors like glare caused by a car or by the sunset directly in front of them.

Current limitations of sensors

As shown by the illustration above, existing systems do not guarantee adequate visibility outside of the red and orange zones, especially in poor visibility conditions. For example, a pedestrian located 20 meters away in the hatched green zone is difficult to detect, as illustrated by the photo above.

Under these conditions, thermal imaging, particularly long-wave infrared, is a complementary solution to existing systems.

          3.2 – Thermal Imaging – this is how it works

Long-wave infrared relies on the natural principle of heat emission, which is captured by thermal imaging sensors to generate real-time images. These sensors are particularly sensitive to infrared radiation emitted by various objects, including pedestrians. The pixels convert this radiation into electrical signals, which, when grouped into matrices, allow for the generation of an image.

According to Planck’s law, the spectral energy density of radiation emitted by a body depends on its emissivity: the higher the emissivity, the more intense the emitted radiation and the shorter the wavelengths it moves towards. Thus, the average human body temperature, which is generally around 37 °C, corresponds to a peak emission of infrared radiation of approximately 9.8 µm.

Thermal imaging sensors, which operate in the long-wave infrared (LWIR) range, are therefore intrinsically efficient at detecting the body heat of pedestrians, even in total darkness, passively and with low power consumtion of about 1W. Thanks to high sensitivity to infrared radiation, thermal imaging represents a significant step ahead for nighttime detection. Indeed, low sensitivity to visible light and the ability to operate in darkness allow for reliable identification of pedestrians under all circumstances, 24/7, even in the darkest environments.

The photos below provide proof points of how a VGA thermal imaging sensor contributes to the current RGB normative system.

        . Without thermal imaging: the pedestrian is only visible at 30 km/h.

        . Adding thermal imaging, the pedestrian is visible at 90 km/h.

          3.3 – LYNRED: The Reference Partner

Since 40 years, LYNRED is the recognized expert in designing and manufacturing a wide range of thermal imaging sensors and modules. The company partners with automotive suppliers to develop reliable and effective solutions capable of detecting and classifying obstacles in all lighting and weather conditions, thereby ensuring the safety of vehicles and their surroundings.

Thermal imaging technology enables visibility in a variety of situations, whether it is darkness, sunlight, headlight glare, shadows, or even fogs, at any time of day or night. When combined with machine learning algorithms for object classification, this technology generates essential data from the long-wave infrared part of the electromagnetic spectrum, thus enhancing vehicle decision-making in environments where other solutions shows to be less effective.

Moreover, thermal imaging shows to be a promising alternative to LIDAR in challenging visibility conditions, overcoming its limitations through the use of infrared radiation that produces clear and precise images, even in the presence of visual obstacles. Thanks to continuous technological advancements, this technology plays an increasingly prominent role in various fields, enhancing safety, monitoring, and the performance of autonomous systems, even in difficult environmental conditions.

Eager to meet international standards, LYNRED already holds ISO 9001, EN9100, and ISO 14001 certifications. Our teams are currently working towards acquiring the automotive-specific IATF 16949 certification aimed at establishing a quality management system that encourages continuous improvement, defect prevention, and the reduction of non-conformities and scrap in the supply chain.

           3.3.1 – Chosing the right format

There is a demand on the market for increased performance and robustness of autonomous systems. Facing the new requirements, existing systems risk to fail during the homologation phases or generating phantom braking during road use. Thermal imaging provides a threefold benefit:

• Reducing false positives and therefore directly reducing phantom braking, while also addressing false negatives. These factors are crucial for successfully passing homologation tests. While it may be easy to excel in one area, it is often challenging to perform well in both aspects, which is what thermal imaging achieves with an Average Precision benefit of 36% compared to visible light alone.

• Finally, thermal imaging stands out by offering early, confirmed detection, enabling anticipation of situations and triggering avoidance or smooth braking, thus improving the driving experience.

           3.3.2 – Going further with LYNRED

LYNRED has developed the first open-source, large-scale European thermal image dataset dedicated to the automotive industry, advanced driver assistance systems (ADAS), and other artificial intelligence (AI)-based applications. The LYNRED Mobility Dataset is the tool to use for testing effectiveness of thermal imaging.

The unique LYNRED Mobility Dataset includes over 250,000 thermal images, providing AI researchers with an unprecedented resource to assess the benefits of thermal technology for mobility applications and more.

Captured over several years and seasons using a variety of thermal cameras and visible cameras, the LYNRED Mobility Dataset offers a highly diverse and realistic set of road traffic scenarios. It is designed to help AI models learn to detect obstacles and respond to the complexities of traffic environments – from pedestrians crossing snowy rural roads to vehicles navigating through urban night scenes.

This dataset features three complementary functionalities: 

• Multimodal detection: to train AI algorithms with up to nine classes under various weather conditions,
• Stereovision: to merge multimodal data, stereo thermal IR, and stereo RGB visible, enabling tracking (video sequences) with perfectly synchronized images,
• Distance estimation: to estimate the detection distance of pedestrians under different automatic emergency braking system conditions, beyond regulatory requirements.

Discover the LYNRED Mobility Dataset in detail on our website: https://www.lynred.com/lynred-mobility-dataset

In conclusion, the FMVSS 127 regulation is boosting innovation in the field of automotive sensors and triggers a crucial step towards improving safety standards. In this context, thermal imaging is positioned as an essential technological complement, enabling to see where visible cameras and radars fail (total darkness, glare, etc.) while minimizing the risk of false positives.

Contact us today to discuss your needs and discover how thermal imaging can enhance the reliability of your PAEB systems.