Apple has been working on presence detection for Macs and a future TV since 2005 (published in 2009). Our cover graphic is from that patent. Over the years Apple’s R&D teams have filed other patents covering user presence systems ( 01, 02, 03 and 04). On Thursday, the US Patent & Trademark Office published yet another patent application from Apple that relates to improved wake-up and authentication methods or systems for electronic devices that uses machine learning and a special camera to detect presence and user ID.
Apple’s invention covers devices, systems, and methods for detecting movement in a physical environment that corresponds to a movement (e.g., a person approaching an electronic device) based on image data (e.g., event data).
In some implementations, the image data used to detect a movement (e.g., an approaching person) enables detecting a person but does not able detection (e.g., identification) of a specific person.
In some implementations, the one or more image sensor systems are configured to obtain image data that corresponds to at least a portion of the face of the user that includes the eyes of the user.
In some implementations, a single image sensor includes both an arrangement of first pixels and a plurality of second pixels on or near a perimeter of the arrangement (e.g., in a surrounding rectangle) that generate the event data.
In some implementations, detecting a person approaching the electronic device activates a user verification process at the electronic device.
In some implementations, the plurality of second pixels on the perimeter of the arrangement of first pixels constantly monitor the physical environment, and the user verification process is activated upon detection of a movement such as a person approaching the electronic device.
Apple’s patent FIG. 3A below covers a MacBook that has a sensor with a standard frame camera portion and a plurality of dynamic vision sensor (DVS) pixels (or event camera). In some implementations, the sensor is used to detect a user of the electronic device. In some implementations, the user detection uses a low power configuration that enables a start-up screen on the electronic device.
As shown in FIG. 3A above, an arrow indicates a field of view (FOV) of the sensor #330. In some implementations, the FOV of the event camera portion #334 follows at least one edge of the FOV of the frame camera portion.
Apple’s patent FIGS. 5A-5B below are block diagrams illustrating event data of a person detected by event camera pixels in a sensor.
As shown in FIG. 5A above, event data #500A is caused by the person moving in side-to-side in front of the sensor having a rectangular donut of event camera pixels in the event camera portion 334. As shown in FIG. 5B, event data 500B is caused by the person moving in top-to-bottom in front of the sensor 330 having the rectangular donut of event camera pixels in the event camera portion 334.
In some implementations, movements of objects into the FOV of the sensor (camera #330) cause contrast changes that are detected by pixels as events in the event camera portion that are determined over a time period and analyzed.
In some implementations, the temporal event data are designated for automated processing (e.g., machine viewing and not human viewing) to address privacy concerns.
In some implementations, the patterns of movements of detected objects in the event data determine whether the detected moving object matches or correlates to speed, size, movement characteristics, or shape characteristics of an approaching person.
In some implementations, the patterns of movements of detected objects in the event data are compared with known profiles of approaching person movements. In some implementations, the patterns of movements of detected objects in the event data are compared using frequency domain representations (e.g., Fourier transform) that may repeat as a function of time.
In some implementations, the patterns of movements of detected objects in the event data are compared using Machine Learning (ML). ML methods for object detection include machine learning-based approaches or deep learning-based approaches.
To review the finer details of this invention, see Apple’s patent application number 20210373643 was published today by the U.S. Patent Office.
Considering that this is a patent application, the timing of such a product to market is unknown at this time.
Originally published at https://www.patentlyapple.com.