Evaluation of deep learning models for aerial object detection using RGB and thermal modalities

Real-time detection of people from aerial images is important for many uses, including security surveillance, crowd monitoring, search-and-rescue, traffic control, and disaster response. The traditional deep learning-based small object detectors are designed to utilize texture-rich information in RGB imagery. Over the years of exploration, it is also clear that RGB-based detectors have not performed equally well when the imagery lacks illumination information. Recently, aerial object detection using thermal-infrared (TIR) imagery has demonstrated that thermal signals, being independent of visible light, can significantly aid detection in low- light or night-time conditions. Detection using RGB-TIR image fusion has been conducted, examining the advantages of both modalities to enhance detection effectiveness in both daytime and nighttime situations. Minimal work has been carried out comparing the object detection evaluation details across RGB and thermal modalities. Given this, we have analyzed the performance evaluation of object detection in RGB and TIR images separately for several advanced detectors, including Faster R-CNN, Cascade R-CNN, and DDOD. The results from our experimental study indicate that object detectors trained in TIR imagery outperform those trained on RGB images. Among the detectors using TIR images, Faster R-CNN achieves the best performance, followed by DDOD and Cascade R-CNN. In contrast, for detectors trained on RGB images, DDOD achieves the highest accuracy, with Faster R-CNN and Cascade R-CNN ranking second and third, respectively.