A Reconfigurable Fuzzy-Logic Audio-Visual Fusion Implementation for Area Threat Identification

The rapidly developing threat due to misuse or mishaps from low-flying aerial vehicles in the last few years is extremely concerning. The threat may even extend beyond military and can threaten the lives of ordinary civilians in a bid to create chaos and confusion. Therefore, there is a need for a cost-effective system that can be deployed in mass at a variety of civilian and military settings. This study addresses this concern and proposes a novel, reconfigurable audio-visual area detection system using fuzzy logic-based sensor fusion, tailored for both static and mobile detection scenarios. The architecture fuses image and sound data to identify threats by integrating a YOLO11(You Only Look Once) based visual detection model with a lightweight CNN (Convolutional Neural Network) for audio classification. This detection technique leverages FPGA (Field Programmable Gate Array) based hardware for efficient real-time deployment in edge environments. The Audio-Visual (Multimodal) inputs are merged through a fuzzy inference model for better robustness, accuracy, mainly for adverse environmental, competitive, and noisy conditions. The system demonstrates over 97% accuracy on test data and maintains competitive performance on unseen datasets. The study looks into the comparison of YOLOv5 with YOLO11 and the advantage of using YOLOv5 for the deployment of CNN on Kria KV260, including limitations of using it. Furthermore, an FPGA-based fuzzy motor controller for movable drone detection shows substantial improvements in response time, energy efficiency, and adaptability over traditional PID (Proportional-Integral-Derivative) controllers. This work presents a scalable and low-power drone detection solution applicable to both defense and civilian airspace safety.