BREE-HD: A Transformer-Based Model to Identify Threats on Twitter

With the world transitioning to an online reality and a surge in social media users, detecting online harassment and threats has become more pressing than ever. Gendered cyber-hate causes women significant social, psychological, reputational, economic, and political harm. To tackle this problem, we develop a dataset and propose a transformer-based model to classify tweets into threats or non-threats that are either sexist or non-sexist. We have developed a model to identify sexist and non-sexist threats from a collection of sexist, non-sexist tweets. BREE-HD performs extraordinarily well with an accuracy of 97% when trained on the dataset we developed to detect threats from a collection of derogatory tweets. To provide insight into how BREE-HD makes classifications, we apply explainable A.I. (XAI) concepts to provide a detailed qualitative analysis of our proposed methodology. As an extension of our work, BREE-HD could be used as a part of a system that could detect threats targeting people specifically tailored to classify them in real-time adequately.