Detecting Malicious DNS over HTTPS Traffic Using Machine Learning

Network with the internet has grown-up very faster compared with any other technology around the world. From the beginning of the Internet, the Domain name system (DNS) is an integral and important part of it. The primary task of DNS is to redirect the users at correct computers, applications, and files by mapping IP and domain name. Due to certain security flaws of DNS, it is always a major attack target for attackers like DNSbased malware, DNS-amplification, false-positive triggering, DNS tunneling, etc. DNS over TLS (DoT) and DNS over HTTPS (DoH) are recently developed and deployed by Google and Cloudflare to prevent these types of attacks. DoT and DoH are the standard protocols which mainly designed for privacy and security by encrypting the DNS traffic between users and DNS resolver servers. This paper uses various machine learning classifiers such as (i) Naive Bayes (NB), ii) Logistic Regression (LR), iii) Random Forest (RF), (iv) K-Nearest Neighbor (KNN), and (v) Gradient Boosting (GB) to detect the malicious activity at DNS level in the DoH environment. The experiments are conducted on a benchmark MoH dataset (CIRA-CIC-DoHBrw-2020). Several features are used to develop a robust model. The experimental outcome confirmed that the RF and GB classifiers are better choices for the said problem. Since, majority of the malicious activity detected by the developed model, it can be said that the ML-based algorithms are a better option for the prevention of DNS attacks on DoH traffic.