According to foreign media reports, the US Patent & Trademark Office announced a patent application from Apple Inc., which is related to its Project Titan, specifically a multimodality. Sensing system for detecting and recognizing objects while driving autonomous vehicles at night. Compared with traditional car headlights, the new system has a more obvious advantage when driving at night, saving more lives in the future.

 (Source: Apple Patent Website)

Some automation systems need to collect and process large amounts of sensor data to identify objects in their surroundings. In order to facilitate navigation and robust control of the automation system, processing sensor data is often subject to real-time constraints, and this is what Apple is trying to improve. In summary, Apple's invention involves the use of multi-modal sensing systems to detect and identify objects while driving autonomous vehicles.
Apple explained that night or low-light environments can challenge the control system of autonomous vehicles. For example, the level of illumination of automotive headlamps at night is limited by laws and regulations, which limits the effective detection range of visible light sensors (such as cameras), which are commonly used to detect objects in or near vehicle paths. Limited effective detection range (eg, approximately 60 meters, 180 feet) will affect object detection and classification, which will reduce vehicle safety and reduce the speed of safe vehicle travel.

Combined with a variety of complementary image sensing technologies, it is possible to solve the problem of object detection and classification in night or low light environments. For example, near-infrared lights can be installed on cars, and laws and regulations may require more or less restrictions on the level of illumination of such lights.

In addition, a near-infrared sensor with near-infrared illumination can be mounted on the vehicle to capture high-resolution image information of objects in or near the vehicle path and at longer distances (eg, 200 meters, 600 feet) from the vehicle. . In this way, objects can be detected and sorted earlier as the vehicle travels, thereby improving safety and increasing maximum speed. Near-infrared illumination projects near-infrared light into a relatively narrow field of view (eg, a 30-degree cone angle).

Although the detection range of visible light sensors is relatively limited, it can provide high-resolution image data of objects of various colors (such as red, green and blue). In addition, visible light sensors can provide a wider field of view (such as a 120-degree field of view). Explore the path ahead of the vehicle.

The Long Wave Infrared Sensor (LWIR) captures the heat radiation naturally generated by the object from the surroundings of the vehicle and therefore does not rely on the light. The effective detection range of long-wave infrared sensors may be limited, limited by the resolution of the sensor and the resolution requirements for object detection and classification. The long-wave infrared sensor includes a large number of component sensors that provide a wide field of view (such as a 180-degree field of view) to detect the surroundings of the vehicle. In addition, long-wave infrared sensors provide images of objects in lower resolution environments.

In some operations, the objects detected by the long-wave infrared sensor (the detection of such objects based on low-resolution image data) can be classified by adjusting the control parameters of other sensing methods and image processing resources, thereby allowing the vehicle to Computer vision resources are concentrated in areas of interest associated with detecting objects.

Apple described in the patent application that the system has been improved compared to previous self-driving vehicle computer vision systems. Some operations can increase the effective detection range, detect and classify objects in or near the vehicle path; some operations can classify objects more accurately in low-light environments, thereby improving the safety of autonomous vehicle control systems, and Improve the maximum safe speed of vehicles in low light conditions.

Source: Geshi Automobile