ModernBERT-XAI: a synergistic approach to sentiment analysis with layer-wise learning and SHAP-LIME interpretability

Sentiment analysis, a critical task in natural language processing (NLP), has seen remarkable advancements with transformer-based models like BERT. However, the opacity of these models poses challenges in domains requiring transparency and interpretability. This paper addresses this gap by fine-tuning ModernBERT, an enhanced variant of BERT, on the IMDb movie review dataset for sentiment classification, achieving state-of-the-art performance. To improve interpretability, we integrate explainable AI (XAI) techniques, including SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations), which provide insights into the model's decision-making process. Our methodology incorporates layerwise learning rate decay to optimize fine-tuning, ensuring that lower layers retain general knowledge while higher layers adapt to the sentiment analysis task. Comprehensive evaluations demonstrate the model's outstanding performance, achieving an accuracy of 95.78%, precision of 95.10%, recall of 96.52%, and an F1-score of 95.81%. The ROC curve further confirms the model's robustness, achieving an AUC of 0.9904. SHAP and LIME analyses reveal the contribution of key words and phrases to sentiment predictions, enhancing transparency and trustworthiness. By combining high performance with interpretability, this work paves the way for transparent and trustworthy AI in sentiment analysis, benefiting domains like healthcare, market research, and social media monitoring.