According to the report of the company, the US Patent and Trademark Office recently announced an Apple patent application for the self-driving car Titan project "Project Titan", which proposed a multi-mode for the detection and recognition of autopilot objects at night. Sensing system. When driving at night, Apple's new system's superior performance over traditional car headlights will greatly improve driving safety and avoid traffic accidents that cause casualties.

Currently designed autopilot systems mostly detect and identify objects in the surroundings of the vehicle by collecting and processing large amounts of data captured by various sensors. Automated vehicles need to process sensor data as much as possible in real time for automatic navigation and robust control of the automation system. This is the direction that Apple's invention patent is trying to improve.

Overall, Apple's invention patents propose a multi-mode sensing system and its application that enables autonomous vehicles to detect and identify objects in their surroundings during nighttime driving.

Apple introduced in the patent that the night or low illumination environment poses a challenge to the control system of autonomous vehicles. For example, the relevant regulations impose restrictions on the illumination level of vehicle headlights at night, which may correspondingly limit the effective detection range of visible light sensors (such as cameras), which are commonly used for the detection and identification of obstacles.

Detecting and classifying the limits of the effective range of obstacles (eg, about 60 meters) will reduce the safety of self-driving vehicles and reduce the maximum speed at which vehicles can travel safely.

Near infrared (NIR) sensing

The combination of multiple complementary image sensing technologies can effectively solve the problem of detecting and classifying objects in a night or low illumination environment. For example, current regulations on the illumination level of vehicles near-infrared illuminators may be loose or unrestricted.

Near-infrared illuminators with near-infrared sensors capture high-resolution images of objects in or near the vehicle's path and can significantly increase the effective detection range (for example, up to 200 meters).

The larger effective detection range enables the vehicle to detect and classify obstacles earlier in driving, improve driving safety, and indirectly increase the maximum speed of autonomous driving. Near-infrared illuminators can project near-infrared light in a relatively narrow field of view (eg, a 30 degree sector).

Although the detection range of visible light sensors may be relatively limited, they may provide high resolution image data in multiple color channels (eg, red, green, and blue). In addition, the visible light sensor can also provide a wider field of view (eg, a 120 degree field of view).

Long wave infrared (LWIR) sensing

The long-wave infrared sensor captures the heat radiation emitted by the object itself in the surroundings of the vehicle, so there is no need to rely on the illuminator. The effective detection range of long-wave infrared sensors is mainly limited by the resolution of the sensor and the resolution requirements of object detection and classification.

Long-wave infrared sensors and their arrays provide a 180-degree wide field of view around the vehicle. Long wave infrared sensors provide relatively low resolution images of objects in the environment surrounding the vehicle.

In some embodiments, the object detection based on the low-resolution image of the long-wave infrared sensor, by adjusting the control parameters of other sensors, and/or the provision of image processing resources, focuses the computer vision processing resources of the vehicle on the object being detected. The area of interest to achieve classification of the object being detected.

In addition, the power level or field of view of the illuminator (eg, near-infrared illuminator or visible illuminator) can also be adjusted to enhance imaging of the region of interest. It is also possible to adjust the calculation control parameters and apply them to the portion of the image associated with the region of interest.

The technology described in this patent application by Apple, or an improvement over the computer vision system of existing autonomous vehicles, can improve the detection and classification of the effective range of objects around the vehicle; it can be more accurately in low illumination environments. Classification of objects; improved safety of autonomous vehicle control systems and increased maximum safe driving speed in low illumination environments.
The following figure is a block diagram of the multi-mode sensing system used by Apple in this patent for night-time autopilot vehicle object detection and recognition, including the field of view of many different types of sensors mounted on the vehicle. In the figure, 710 denotes an autonomous driving vehicle; 720 sectors denote a field of view of a visible light image sensor (VISIBLE FOV); 730 a plane representing a field of view of a long-wave infrared sensor (LWIR FOV, maximum angle of view); 740 denotes a view of a near-infrared image sensor Field (NIR FOV), the field of view is slightly smaller, but the effective detection distance is the largest.

Titan plan is dead? I believe this is Apple's uncompromising quality and high standards.

Apple's patent application in the third quarter of 2018, it is not yet certain when its self-driving car will come out.

As early as 2014, Apple launched the Titan self-driving car project, Titan is the Protoss who ruled the world in Greek mythology. Apple's initial goal was to create a car that could subvert the industry, and the product should be unveiled in 2019 as planned. But in 2016, Apple suddenly cut a large number of employees of the project, and layoffs again in early 2019. These behaviors have to be suspected of a problem with the autopilot project or Apple's problem.

According to foreign media analysis, one of the possible reasons is that Apple's mobile phone market sales are sluggish, and Apple has to cut expenses for non-core programs.

In addition, Apple officially announced the abandonment of the AirPower wireless charging board project. Apple officially said: "After a lot of efforts, we confirmed that AirPower can not meet our high standards, we canceled the project." The safety of autonomous vehicles is about life, Apple's product philosophy and high standards, may be current The level of design and manufacturing has not been able to keep up. Therefore, Apple's uncompromising product quality may be a potential cause of Titan's plan to confuse.