How does face recognition work?

We encounter image processing every day. Often we are probably not even aware of this.

How many of you unlock your cell phone with your face? – Probably a few. And this has probably been the case for several years.
But how many of you know what is actually behind facial recognition and how it works? Caught? – Don’t worry, you’ll find out here!

As soon as you calibrate the face recognition in your phone, a mathematical model is created based on your face. Several factors play a role here. For example, the distance between your eyes or ears, the depth of your eyes and many others. Each software takes different anchor points into account. There can be up to 80 different anchor points in total. When unlocking the phone or logging into apps, the face is then compared with the saved model. And if it matches, the phone can be unlocked. So far, so good.

Incidentally, facial recognition has been around much longer than smartphones were even suitable for mass use and facial recognition is used several times a day, even in the private sphere. In the early days, facial recognition was used to identify criminals, among other things. This was not always unproblematic, as the technology was far from perfect and false faces were also recognized. It was only with the further development of artificial intelligence and neural networks that false-positive faces became much rarer.

Today, facial recognition technology in smartphones comprises several components, including an infrared camera and a dot projector. When a user looks at their device, the dot projector sends a pattern of infrared dots onto their face. The infrared camera then captures an image of this pattern and sends it to the chip, where it is processed by neural networks to create a 3D representation of the user’s face. This 3D model is then converted into a mathematical representation called a depth map, which is a collection of data points that represent the contours and shape of the user’s face. The depth map is then fed into a neural network, a type of artificial intelligence algorithm designed to learn from data. The neural network is trained using a large data set of faces in order to recognize and classify various facial features, such as the aforementioned anchor points.

Face registration is also already underway in industries such as security, retail and healthcare. Methods are also being tested in vehicles such as cars to detect microsleeps at an early stage. While the underlying technology and mathematical models used in these industries may be similar to those explained previously, there may be some differences in the way they are implemented and the level of accuracy required. That depends on the respective environment.