Facial recognition technology is used in many different areas – from airports, retail stores, healthcare, law enforcement and marketing to banks – to identify people through images and videos of their faces. However, a drawback of this technology is that it does not accurately recognise the faces of people with highly pigmented skin tones.
“One way to enhance the facial recognition of people with darker skin is to incorporate the portion of the infrared spectrum (the near-infrared part) that electronic sensors can perceive,” says Alex Muthua, who received his Master’s degree in Electronic and Electrical Engineering on Wednesday (7 December 2022) at Stellenbosch University’s December graduation.
As part of his study, Muthua explored how infrared light could help to improve the facial recognition of people with a highly pigmented skin tone.
“Individuals with highly pigmented skin appear darker in visible light images, which means that there is less light information available in the image on which to perform facial recognition. By adding infrared, one can increase the dynamic range of the intensity values, which in turn aids the face recognition system.”
According to Muthua, most digital cameras have a filter that blocks out the infrared light needed for this type of facial recognition. Using a camera that comes with the filter already removed, Muthua took 9 000 images of 500 individuals with highly pigmented skin in different light spectra – visible, near-infrared and a combination of the two (full-spectrum).
In this way, he was able to augment existing datasets with images of highly pigmented individuals. Muthua points out that there is typically a bias in these datasets, with more images of light skin pigmentation represented as opposed to darker skin pigmentation. These datasets also don’t contain infrared images.
He also assessed the impact of narrow and wide cropping, different facial orientations, and sunlight and shaded conditions.
“We found that by using infrared light, the faces of people with a highly pigmented skin tone could be recognised with greater accuracy compared to using only visible light,” says Muthua.
He also fine-tuned an existing face recognition algorithm and inspected the activation maps of an available convolutional neural network, which is a machine learning architecture used primarily for image processing. This was done to determine the features that are the most important for facial recognition for people with highly pigmented skin.
“We found that the nose area appears to be the most important feature for facial recognition compared to the chin and forehead.”
Muthua says as face recognition technology continues to grow, its performance must be improved because any bias or disparity in this regard may be detrimental to its advancement and potential use.
- Photo: Alex Muthua at the graduation ceremony. Photographer: Stefan Els
