2D-3D Facial Image Analysis for Identification of Facial Features Using Machine Learning Algorithms with Hyper-parameter Optimization for Forensics Applications

The Face Recognition System is a remarkable process that humans naturally use, as a precursor to communicating nonverbal cues in their normal daily lives. Outside of this regime, face recognition systems are increasingly being used for recognizing faces in commercial and law enforcement applications. Currently, it is one of the most sought-after detection methods used in forensics for criminal identification purposes. Owing to similarities in the appearance of certain faces, face recognition systems today face substantial challenges in reliably identifying them in settings that require high detection rates. In addition, among criminal suspects found to have similar faces, this problem poses a great challenge in implementing forensic illegal and fraudulent activity detection. This study addresses the development of a detection pipeline that is data-driven, and fast, and incorporates structural information to overcome some of these issues. The novelty of this study lies in reconstructing a three-dimensional face mesh from two-dimensional images, of located 468 landmarks employing the media pipe framework. This two-dimensional to three-dimensional annotation provides a higher quality of three-dimensional reconstructed face models without the need for any additional three-dimensional morphable models. The proposed approach works better for detecting multiple faces in real-time and even in challenging uncontrolled conditions such as large pose variations, expression variations, and occlusion variations. The dataset consists of facial images collected from the web which provides the correct matching decisions in an unconstrained environment that is then used in forensic sciences to be presented as statistical evidence. Quantitative similarity measures are used as inputs for various classifiers to identify criminals in forensic investigations. The proposed methods were validated and tested to achieve comparable recognition performance and hint at the potential of further research for scale-up implementation.