Mediapipe offers very efficiant and accurate face detection models for detecting multiple faces from images. It also provides 6 facial landmarks with detections. It is very simple to use like other mediapipe models and runs efficiently on modern cpus. First, you need to install mediapipe python package for getting started on face detection.

pip install mediapipe

Now, we will use opencv to read images and provide as input to mediapipe for face detection. 

import cv2
import mediapipe as mp

Now we need to initialize a mediapipe face detection model and we will also use mediapipe drawing utils to easily draw points and rectangles on image. Here we provide different options while creating a face model object.

  • model_complexity Could be either 0 for short range and 1 for full range where objects are at some distance.
  • min_detection_confidence Detection results confidence, could be adjusted according to requirements and input image

Now we will get mediapipe drawing utils and load a model.

mp_drawing = mp.solutions.drawing_utils
# load face detection model
mp_face = mp.solutions.face_detection.FaceDetection(
    model_selection=1, # model selection
    min_detection_confidence=0.5 # confidence threshold
)

Read image from directory using opencv and provide as input to model. Once we get results, we can iterate over each detection and draw on image.

image = cv2.imread('image0.png')
image_input = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# get results
results = mp_face.process(image_input)

if not results.detections:
    print('No faces detected.')
else:
    for detection in results.detections: # iterate over each detection and draw on image
      mp_drawing.draw_detection(image, detection)

cv2.imwrite('image0-pred.jpg', image)

Here are results for given image. It could miss results in some cases where background is dark, but we can achieve results by changing threshold for some scenarios.

My alt text

Crop Faces

We can also process detection data ourself and can use custom functions to draw on image and also extract faces from images. By default mediapipe returns detection data in normalize form and we have to convert to original size by multiplying x values by width and y values by height of input image.

First, we extract all points from results and save in a list after processing those points.

detection_results= []
for detection in results.detections:
    bbox = detection.location_data.relative_bounding_box
    bbox_points = {
        "xmin" : int(bbox.xmin * width),
        "ymin" : int(bbox.ymin * height),
        "xmax" : int(bbox.width * width + bbox.xmin * width),
        "ymax" : int(bbox.height * height + bbox.ymin * height)
    }

    detection_results.append(bbox_points)

This data can be used for custom drawing functions like opencv and other libraries or for extraction of faces from images. For other landmarks, we can also extract those data by extending previous example.

# Facial keypoints labels
FACIAL_KEYPOINTS = mp.solutions.face_detection.FaceKeyPoint

for detection in results.detections:
    # Landmarks
    keypoints = {}
    for kp in FACIAL_KEYPOINTS: # iterate over each landmarks and get from results
        keypoint = mp.solutions.face_detection.get_key_point(detection, kp)
        # convert to pixel coordinates and add to dictionary
        keypoints[kp.name] = {"x" : int(keypoint.x * width), "y" : int(keypoint.y * height)}
        
    # bbox data
    bbox = detection.location_data.relative_bounding_box
    bbox_points = {
        "xmin" : int(bbox.xmin * width),
        "ymin" : int(bbox.ymin * height),
        "xmax" : int(bbox.width * width + bbox.xmin * width),
        "ymax" : int(bbox.height * height + bbox.ymin * height)
    }

    # add score, bbox and keypoints to detection_results
    detection_results.append({
        "keypoints" : keypoints,
        "bbox" : bbox_points,
        "score" : detection.score
    })

Now, this data can be utilized by an image processing library to process and draw on image or input to face recgonition models. For more info on mediapipe for face detection, view documentation at their webpage.

https://google.github.io/mediapipe/solutions/face_detection.html