How Facial Recognition system works

Facial recognition is a way of recognizing a human face through Deep Learning algorithm. A facial recognition system uses bio-metrics to map facial features from an image or video. It compares the information with a database of known faces to find a match. 

How face recognition works:

So how does facial recognition work? Technologies vary, but here are the basic steps:

Step 1. A picture of your face is captured from a photo or video. Your face might appear alone or in a crowd.

Step 2. Facial recognition software reads the geometry of your face. Key factors include the distance between your eyes,height and width of the face and the distance from forehead to chin. The software identifies facial landmarks —that are key to distinguishing your face. The result: your facial signature.

Step 3. Your facial signature — a mathematical formula — is compared to a database of known faces.

Step 4. A determination is made. Your face-print may match that of an image in a facial recognition system database.

Let’s read about step-2 and step-3 in more detail:

Basically there are two major steps of facial recognition:

  1. Face Detection
  2. Face Recognition/Identification or Comparison

Face Detection:

The face detection work to detect faces in an image. Here we work on OpenCV for Face Detection. Although there are other methods also like: Neural Networks, MATLAB etc. Now lets see how the face detection operates:

  • The image processing is done, in which the conversion of RGB to Grayscale, resizing, blurring, sharpening etc. of the images done if needed.
  • The next step is image segmentation, which is used for contour detection. It segments the multiple objects in a single image so that the classifier can quickly detect the objects and faces in the picture.
  • In the next step we have used Haar-Like features algorithm. This algorithm is used for finding the location of the human faces in a frame or image. All human faces have some universal properties like the eyes region is darker than its neighbor pixels or nose region is brighter than eye region.

The haar-like algorithm is also used for feature extraction for face in an image, with the help of filters i.e. edge detection, centre detection, line detection for detecting eyes, nose, mouth, etc. in the picture. It is used to select the essential features in an image and extract these features for face detection. For instance, suppose we wish to identify whose face is present in a given image, there are multiple things we can look at as a pattern:

  • Height/width of the face.
  • Height and width may not be reliable since the image could be rescaled to a smaller face. However, even after rescaling, what remains unchanged are the ratios – the ratio of height of the face to the width of the face won’t change.
  • Color of the face.
  • Width of other parts of the face like lips, nose, etc.

Different faces have different dimensions of facial landmark. Similar faces have similar dimensions. The challenging part is to convert a particular face into numbers – Machine Learning algorithms only understand numbers. This numerical representation of a “face” (or an element in the training set) is termed as a feature vector.

Essentially, given an image, we can map out various features and convert it into a feature vector like:

Height of face (cm)Width of face (cm)Average color of face (RGB)Width of lips (cm)Height of nose (cm)
23.115.8(255, 224, 189)5.24.4
Feature Vectors

Of course there could be countless other features that could be derived from the image (for instance, hair color, facial hair, spectacles, etc). 

Face Recognition/Identification/Verification

Now, once we have encoded each image into a feature vector, the problem becomes much simpler. Clearly, when we have 2 faces (images) that represent the same person, the feature vectors derived will be quite similar. Put it the other way, the “distance” between the 2 feature vectors will be quite small.

We need to learn a high-dimensional feature space which is capable of distinguishing between faces of different persons. Once we have a trained model, we can use it to generate unique features for each face. Finally, we can compare features of new faces with that of known faces to identify the person.