# Not Afraid of the Dark: NIR-VIS Face Recognition via Cross-spectral Hallucination and Low-rank Embedding

**Authors:** Jose Lezama, Qiang Qiu, Guillermo Sapiro  
**arXiv:** [1611.06638](https://arxiv.org/abs/1611.06638)  
**Date:** 2016-11-21  
**Summary page:** https://www.summarizepaper.com/en/paper/1611.06638v1/

## Key points

- Near-infrared (NIR) imaging is commonly used in low-light environments for surveillance technology.
- Matching NIR face images with visible light (VIS) spectrum faces poses a challenge due to limited labeled NIR samples and abundance of VIS face galleries.
- Jose Lezama, Qiang Qiu, and Guillermo Sapiro propose an approach to extend deep learning-based VIS face recognition to the NIR spectrum without retraining existing models.
- The proposed approach includes cross-spectral hallucination and low-rank embedding as key components.
- Cross-spectral hallucination generates synthetic VIS faces from NIR images using deep learning techniques.
- Low-rank embedding restores a low-rank structure within deep features extracted from both NIR and VIS face images.
- Applying either component significantly enhances performance of VIS deep model for cross-spectral face recognition; combining them achieves further improvements without retraining the system.
- State-of-the-art results have been achieved on benchmark datasets like CASIA NIR-VIS v2.0, demonstrating effectiveness in bridging the gap between NIR and VIS face recognition domains.
- This research offers a promising solution for challenges in cross-spectral face recognition by leveraging deep learning techniques to enhance performance across different light spectra seamlessly without extensive retraining efforts.

## Summary

In the field of surveillance technology, the use of near-infrared (NIR) imaging has become increasingly common for capturing images in low-light environments. However, a significant challenge arises when attempting to match NIR face images with those captured in the visible light (VIS) spectrum. This is due to the limited availability of labeled NIR face samples for training and the predominance of VIS face galleries for matching. To address this issue, a novel approach has been proposed by Jose Lezama, Qiang Qiu, and Guillermo Sapiro. Their research focuses on extending the advancements in deep learning-based VIS face recognition to the NIR spectrum without requiring retraining of existing models trained solely on VIS faces. The proposed approach comprises two key components: cross-spectral hallucination and low-rank embedding. Cross-spectral hallucination involves using deep learning techniques to generate synthetic VIS faces from NIR images. On the other hand, low-rank embedding aims to restore a low-rank structure within the deep features extracted from both NIR and VIS face images. Interestingly, it has been observed that applying either hallucination to input NIR images or low-rank embedding to output deep features can significantly enhance the performance of a VIS deep model for cross-spectral face recognition. And when these two components are combined, further improvements in accuracy are achieved without necessitating any retraining of the recognition system. As a result, state-of-the-art results have been obtained on benchmark datasets such as CASIA NIR-VIS v2.0. This showcases the effectiveness of this approach in bridging the gap between NIR and VIS face recognition domains. In conclusion, this research presents a promising solution for overcoming challenges associated with cross-spectral face recognition and highlights the potential of leveraging deep learning techniques to enhance performance across different light spectra. By seamlessly integrating cross-spectral hallucination and low-rank embedding into existing deep models trained on VIS data, this work opens up new possibilities for improving facial recognition systems operating in diverse lighting conditions without requiring extensive retraining efforts.