Facial recognition and biometrics: your smartphone knows you so well / Part 2

What is the use of facial recognition for humans, and companies? How efficient is it. And most importantly, does it put our freedom at risk?

Effective facial recognition tools are used by Internet giants, but also by smartphone manufacturers, especially for unlocking phones, and for recognizing people in buildings.

Towards the democratization of facial recognition

The front camera will first take a picture when you present your face and it will look for your face in the image.

Figure 1: Face search and analysis via a smartphone

https://stock.adobe.com/search?k=phone%20features

Once the position of the face is found, different attributes are taken into account:

- Face shape

- Position and shape of the ears

- Distance between the eyes

- Position of the eyes

- Nose shape

- Position of the mouth

- Shape of the mouth/lips, etc.

Those values are used to create a pattern. Since the objective of your phone is to verify that you are its owner, it will proceed to an authentication. It will compare the pattern it has just calculated with the one you gave it by registering your face the first time. Depending on the security level, the phone will have a higher or lower recognition rate. As far as identification is concerned, we try to determine who is the person we have just seen.

Figure 2: Face identification in open terrain

https://www.shutterstock.com/fr/g/varuna

For example, if a child is wanted, his or her face could be included in a database of wanted children. Each time a person passes in front of the cameras of a station equipped with a facial recognition system, their face is photographed. This is then analyzed to determine whether the face photographed is that of a wanted person or not, and an extract of the photo will be compared with those in the database of wanted children. If a pattern is close enough to the face, an alert will be sent to an operator (there is a match and therefore identification of an abducted child). However, the human operator must remove any doubt and confirm that the recognition system has not made a mistake.

Figure 3: Identification and second control by the human eye.

Identification et deuxième contrôle par l’œil humain

Should we worry about being identified and tracked around the world?

There are limits... and loopholes

Don't panic, your secret identity will not be revealed so easily. This is not because you want to preserve your privacy, but because, contrary to what movies or video games sell you, it is not that easy to recognize a face in a free environment.

When you teach a system that a person's ears are 4 cm long and 2.5 cm wide, it's with a frontal image taken at a specific distance. If the person turns their head 30°, the ears no longer have the same surface area. It is also important to control the light and the background where you take the picture. If there is a bigger shadow at one place of the nose, you distort the shape of the nose. But when you want to use a system to control a "free" space, you don't control the light or the background. Nor do we control the angle of the face at the time of acquisition and neither  the distance at which people will be at the time of the acquisition of the image. In short, many elements are not controlled. And these can have an impact on the appearance of the person.

When the GAFAMs get involved

GAFAM -Google, Amazon, Facebook, Apple, Microsoft-, the leading companies in the digital world, have developed proprietary algorithms for facial recognition. We find Deepface (a combination of deep-learning and face) at Facebook, Or Face Net for Google, which uses it on Google photo.

These algorithms are based on various learning methods to determine a "normalized" image of your face. To do this, it needs several photos with a rigid expression, and without a pose (otherwise the learning is wrong). The algorithms will extract information from the different photos they are given. They will process them in a statistical way to know better and better how to identify you.

As for any learning system, the more data they have in input, the more they can adjust their accuracy. The pictures you put on the internet or on social networks will be good input data. But don't worry, the 98%+ recognition scores of the latest algorithms must be balanced with the constraints on photo types and their variability. These elements apply on a smaller scale with fingerprint identification.

Technological evolution leads to societal evolution

Facial recognition is a powerful tool for increasing the security of places and devices. Facial recognition tools combined with artificial intelligence tools will lead to major upheavals in the evolution of technology and our relationship to society, security and privacy. This evolution generates new questions from a legislative and ethical point of view.

But facial recognition can also help people suffering from a rare disease such as prosopagnosia, which is characterized by the impossibility of recognizing a familiar face. Great developments are yet to come in the field of personal assistance with various applications that, fortunately, are not limited to security.

Inspired by the research of Dr. Paul Ekman, a pioneer in the field of behavioral science  who inspired the character of the series Lie to me,  we could theoretically (but still only theoretically) read the lies and emotions on faces. But that is for a more distant future !

References :

1.       https://www.thalesgroup.com/fr/europe/france/dis/gouvernement/biometrie/reconnaissance-faciale

2.       https://research.fb.com/wp-content/uploads/2016/11/deepface-closing-the-gap-to-human-level-performance-in-face-verification.pdf

3.       https://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Schroff_FaceNet_A_Unified_2015_CVPR_paper.pdf

4.       https://www.rand.org/pubs/documented_briefings/DB396.html