Apple's new AR patent: can identify food freshness and calories with AR glasses

 There has always been news about Apple's VR/AR devices. Although Apple has launched the AR software development platform-AR Kit, it is still uncertain whether Apple will launch VR/AR-related terminal products, but it is basically OK What is certain is that Apple is making considerable investments in VR/AR-related hardware.

Three new patents

According to detailed information from the US Patent Office, Apple’s three new patented technologies are infrared sensing technology for food detection, adaptive facial interface, and eye-tracking technology for head-mounted devices.

The first patent is an electronic device with beam-steered infrared light sensing. This system can use infrared sensors to collect the spectrum of the target object, by using infrared spectrum, visible light camera image, or Data from other sensors are used to analyze object information.

The potential use of this technology patent is in AR devices. For example, users can use AR devices with this system to recognize food images. Through AR recognition, food information can be analyzed, including the freshness, fat content, and Food type, sweetness, etc., so that the maturity and calorie content of the food can be evaluated.

Of course, the scope of application of this technology is not limited to food. Generally speaking, any type of object, including plants, animals, food, paint, housing construction, automobiles, electronic equipment, furniture, etc., can be analyzed using optical sensors. . In addition, the system can also be used to monitor certain physical indicators of the user, such as breathing conditions. Apple's new AR patent: can identify food freshness and calories with AR glassesThe second patent describes an adaptive face interface system (Head-Mounted Display Unit With Adaptable Facial Interface) that can be used in head-mounted display devices.

According to the patent, Apple's head-mounted device may be divided into two parts, one part is a display device, and the other part is a head-mounted accessory similar to a mask.

The head-mounted accessory can well prevent the entry of external light sources, provide a good protection for the user's eyes to avoid interference from ambient light, and can form a complete set of VR/AR equipment after being connected to the display device. Through this interface system, the user's virtual reality or mixed reality experience will be further enhanced. Apple's new AR patent: can identify food freshness and calories with AR glassesThe third patent relates to eye-tracking technology, which can also be applied to augmented reality and virtual reality devices. 

Apple has previously proposed eye tracking related patents, but this patent announced this time is still somewhat different from the previous patents. This patent has made some improvements to the previous research tracking technology. This patent for eye-tracking technology uses a diffraction grating to receive light based on the previous "hot mirror" technology.

The diffraction grating redirects or reflects part of the infrared light reflected from the user's eyes, while allowing visible light to pass through. The camera captures an image of the user's eyes from the infrared light redirected or reflected by the diffraction grating.

The device using the diffraction grating will be smaller than the device using the hot mirror and can improve the viewing angle of the infrared camera. Apple's new AR patent: can identify food freshness and calories with AR glassesThe future of Apple VR/AR devices can be expected in the past few years. Apple has made frequent developments in VR/AR. In the past few years, Apple has applied for a large number of patents related to VR/AR glasses, many of which have been applied to iPad. Waiting for existing equipment.

In addition, Apple has invested a lot of human resources in VR/AR projects, including some important executives. A few days ago, major media reported that Apple announced that Dan Riccio, it's head of hardware engineering, was transferred to a confidential new project. According to various sources, this new project is related to VR/AR equipment and is responsible for the final supervision of the project. And Mike Rockwell, the vice president of Apple, who was in charge of Apple's VR/AR project before, is still responsible for the daily research and development of VR/AR equipment. It is reported that more than 1,000 engineers have worked hard for this project, which shows that Apple attaches great importance to the project.

According to The Information last week, Apple may be developing an MR head-mounted device that will have 8K ultra-high resolution and cash eye-tracking technology. In terms of hardware, this device has more than a dozen cameras that can capture The movement of the human hand completes the interaction between the human and the virtual world.

The Information said that this device will sell for as much as US$3,000 and will be launched in 2022.

Based on all the current information, it is almost certain that Apple's launch of AR/VR/MR headsets can be said to be on the line, only to be released.

Ten years ago, Apple's iPhone subverted the form of a new generation of computer equipment. This time, in the field of virtual reality, whether Apple can once again become a breaker is worth looking forward to.

This article references:

https://appleinsider.com/articles/21/02/11/apple-refining-ar-and-vr-tech-including-sensors-to-detect-food-freshness

https://www.theverge.com/2021/2/8/22272555/apple-dan-riccio-ar-vr-developmen t

https://www.theinformation.com/articles/new-apple-mixed-reality-headset-details-swappable-headbands-eye-tracking