TY - GEN
T1 - Assessing Bias in Large Language Models
T2 - 2024 IEEE International Conference on Modeling, Simulation and Intelligent Computing, MoSICom 2024
AU - Shetty, Poornima
AU - Shrishma Rao, V. S.
AU - Muralikrishna, S. N.
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - As Large Language Models (LLMs) become more integrated into various applications, their potential to display and amplify biases is a significant concern. This study evaluates the social and gender bias in ChatGPT, Gemini, and Claude AI. Unlike previous studies using pre-existing datasets, we used handcrafted questions to reveal biases more effectively. Basic simple questions provided an unbiased answer, however complex biased questions highlighted a few concerns. ChatGPT had a bias all the time while Gemini had a bias sometimes and the rest of the time it did not, or if it did the bias was different, while Claude AI was mostly unbiased. Thus, this study shows that there is variation in levels of bias in different AI models and thus advocates for enhanced strategies of Bias in AI to address the issue of fairness in the use of AI.
AB - As Large Language Models (LLMs) become more integrated into various applications, their potential to display and amplify biases is a significant concern. This study evaluates the social and gender bias in ChatGPT, Gemini, and Claude AI. Unlike previous studies using pre-existing datasets, we used handcrafted questions to reveal biases more effectively. Basic simple questions provided an unbiased answer, however complex biased questions highlighted a few concerns. ChatGPT had a bias all the time while Gemini had a bias sometimes and the rest of the time it did not, or if it did the bias was different, while Claude AI was mostly unbiased. Thus, this study shows that there is variation in levels of bias in different AI models and thus advocates for enhanced strategies of Bias in AI to address the issue of fairness in the use of AI.
UR - https://www.scopus.com/pages/publications/85219589566
UR - https://www.scopus.com/pages/publications/85219589566#tab=citedBy
U2 - 10.1109/MoSICom63082.2024.10881888
DO - 10.1109/MoSICom63082.2024.10881888
M3 - Conference contribution
AN - SCOPUS:85219589566
T3 - IEEE International Conference on Modeling, Simulation and Intelligent Computing, MoSICom 2024 - Proceedings
SP - 133
EP - 137
BT - IEEE International Conference on Modeling, Simulation and Intelligent Computing, MoSICom 2024 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 9 December 2024 through 11 December 2024
ER -