How Facial Recognition system works

Facial recognition is a way of recognizing a human face with the help of Deep Learning. A facial recognition system uses bio-metrics to map facial features from an image or video and then compares the information with a database of known faces to find a match. 

How face recognition works:

So how does facial recognition work? Technologies vary, but here are the basic steps:

Step 1. A picture of your face is captured from a photo or video. Your face might appear alone or in a sea of people.

Step 2. Facial recognition software reads the geometry of your face. Key factors include the distance between your eyes, height and width of the face and the distance from your forehead to chin. The software identifies facial landmarks —that are key to distinguishing your face. The result: your facial signature.

Step 3. Your facial signature — a mathematical formula — is compared to a database of known faces.

Step 4. A determination is made and your face-print may match that of an image in a facial recognition system database.

Let me expand on step-2 and step-3 in more detail:

Basically there are two major steps of facial recognition:

  1. Face Detection
  2. Face Recognition/Identification or Comparison

Face Detection:

Face detection works to detect faces in an image. Here we are using OpenCV for Face Detection. Although there are other methods such as: Neural Networks, MATLAB etc.

Workings of Face Detection:

  • The image processing is done, where in the conversion of RGB to Grayscale takes place in addition to resizing, blurring and sharpening of images (if needed).
  • The next step is image segmentation, which is used for contour detection. It segments multiple objects in a single image so that the classifier can quickly detect the objects and faces in the picture.
  • Finally we use Haar-Like features algorithm. This algorithm is used for finding the location of the human faces in a frame or image. All human faces have some universal properties, for example, the eye region is darker than its neighbor pixels or the nose region is brighter than the eye region.

The haar-like algorithm is also used for feature extraction of a face in an image, with the help of filters i.e. edge detection, center detection and line detection for detecting eyes, nose, mouth, etc. in the image. For instance, suppose we wish to identify whose face is present in a given image, there are multiple things we can look at:

  • Height/width of the face (Height and width may not be reliable since the image could be rescaled to a smaller face. However, even after re scaling, what remains unchanged are the ratios – the ratio of height of the face to the width of the face won’t change.)
  • Color of the face.
  • Width of other parts of the face like lips, nose, etc.

Different faces have different dimensions of facial landmarks. Similar faces have similar dimensions. The challenging part is to convert a particular face into numbers – Machine Learning algorithms only understand numbers. This numerical representation of a “face” (or an element in the training set) is termed as a feature vector.

Essentially, given an image, we can map out various features and convert it into a feature vector as shown below:

Height of face (cm)Width of face (cm)Average color of face (RGB)Width of lips (cm)Height of nose (cm)
23.115.8(255, 224, 189)5.24.4

This isn’t an exhaustive list. Of course there could be countless other features that could be derived from the image (for instance, hair color, facial hair, spectacles, etc). 

Face Recognition/Identification/Verification

Once we have encoded each image into a feature vector, the problem becomes much simpler. Clearly, when we have 2 faces (images) that represent the same person, the feature vectors derived will be quite similar. Put in another way, the “distance” between the 2 feature vectors will be quite small.

Using deep learning, a high-dimensional feature space which is capable of distinguishing between faces of different persons is essential. Once we have a trained model, we can use it to generate unique features for each face. Finally, we can compare features of new faces with that of known faces to identify the person.