The US Patent & Trademark Office has published a patent application from Apple related to developing a mixed reality headset that uses advanced RGB-D sensors to produce accurate 3D images for virtual reality, augmented reality and mixed reality applications. The patent covers devices, systems, and methods that implement simultaneous localization and mapping for RGB-D sensors, such as RGB-D cameras. 

One possible implementation involves the device receiving multiple frames of a real-world scene within a field of view of a camera at multiple times, the frames including color values and depth values for pixels for the field of view of the camera. The device selects keyframes from the multiple frames of the real-world scene within the field of view of the camera. The keyframes are associated with camera poses defined in a three-dimensional (3D) coordinate system.

The device receives a current frame of the real-world scene currently within the field of view of the camera. The current frame includes current color values and current depth values for the pixels for the field of view of the camera at a current time. The device determines a current camera pose of the camera in the 3D coordinate system based on the current frame. The device then provides a virtual representation of the current frame based on the current camera pose of the camera and two or more of the keyframes.

The virtual representation is provided based>Apple¡¯s use of RBG-D sensors is not entirely new. Those sensors have been used in Microsoft's Kinect, which in the early stages used PrimeSense technology now owned by Apple. The Asus Xtion Pro Live uses RBG-D sensors as well.

Apple patent places biometric sensors in earbuds

A recent Apple patent discusses the possibility of embedding biometric sensors within earbuds to monitor various health conditions. (Pixabay)

Apple¡¯s ubiquitous Ear Pods may someday do more than just provide private listening.

The smartphone and computing devices company has filed a patent for developing ear buds with biometric sensors that could be used to measure heart rate, blood volume, and respiratory rate, according to an article>According to the article, the patent provides details on how earbuds could be configured to include biometric sensors. One sensor could be pressed up against the tragus in the ear to take biometric measurements. For instance, a PPG sensor could illuminate a patch of skin to measure changes in light absorption of the skin. By measuring changes in the light absorption as caused by profusion of blood, the earbuds could measure heart rate, blood volume and respiratory rate.

In another scenario quoted in the article, the patent suggests a wired earbud electrically coupled to another earbud with an electrode can cooperatively measure different biometric parameters. For instance, the electrodes can be configured to measure the galvanic skin response (GSR) of a user, which can help determine the amount of stress being experienced by the user at any time. The patent also describes a scenario where the electrodes can be used to measure more detailed parameters of the heart, by taking the form of an electrocardiogram (EKG) sensor or an impedance cardiography (ICG) sensor.

The article noted that according to the patent, the sensors would be able to determine which ear the earbud has been placed in, and alter its operation accordingly.