TY - GEN
T1 - HindiWSD
T2 - 6th Workshop on Indian Language Data: Resources and Evaluation, WILDRE 2022
AU - Yusuf, Mirza
AU - Surana, Praatibh
AU - Sharma, Chethan
N1 - Publisher Copyright:
© European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0.
PY - 2022
Y1 - 2022
N2 - A lot of commendable work has been done, especially in high resource languages such as English, Spanish, French, etc. However, work done for Indic languages such as Hindi, Tamil, Telugu, etc is relatively less due to difficulty in finding relevant datasets, and the complexity of these languages. With the advent of IndoWordnet, we can explore important tasks such as word sense disambiguation, word similarity, and cross-lingual information retrieval, and carry out effective research regarding the same. In this paper, we worked on improving word sense disambiguation for 20 of the most common ambiguous Hindi words by making use of knowledge-based methods. We also came up with “hindiwsd”, an easy- to-use framework developed in Python that acts as a pipeline for transliteration of Hinglish code-mixed text followed by spell correction, POS tagging, and word sense disambiguation of Hindi text. We also curated a dataset of these 20 most used ambiguous Hindi words. This dataset was then used to enhance a modified Lesk algorithm and more accurately carry out word sense disambiguation. We achieved an accuracy of about 71% using our customized Lesk algorithm which was an improvement to the accuracy of about 34% using the original Lesk algorithm on the test set.
AB - A lot of commendable work has been done, especially in high resource languages such as English, Spanish, French, etc. However, work done for Indic languages such as Hindi, Tamil, Telugu, etc is relatively less due to difficulty in finding relevant datasets, and the complexity of these languages. With the advent of IndoWordnet, we can explore important tasks such as word sense disambiguation, word similarity, and cross-lingual information retrieval, and carry out effective research regarding the same. In this paper, we worked on improving word sense disambiguation for 20 of the most common ambiguous Hindi words by making use of knowledge-based methods. We also came up with “hindiwsd”, an easy- to-use framework developed in Python that acts as a pipeline for transliteration of Hinglish code-mixed text followed by spell correction, POS tagging, and word sense disambiguation of Hindi text. We also curated a dataset of these 20 most used ambiguous Hindi words. This dataset was then used to enhance a modified Lesk algorithm and more accurately carry out word sense disambiguation. We achieved an accuracy of about 71% using our customized Lesk algorithm which was an improvement to the accuracy of about 34% using the original Lesk algorithm on the test set.
UR - https://www.scopus.com/pages/publications/85146277247
UR - https://www.scopus.com/pages/publications/85146277247#tab=citedBy
M3 - Conference contribution
AN - SCOPUS:85146277247
T3 - 6th Workshop on Indian Language Data: Resources and Evaluation, WILDRE 2022 - held in conjunction with the International Conference on Language Resources and Evaluation, LREC 2022 - Proceedings
SP - 18
EP - 23
BT - 6th Workshop on Indian Language Data
A2 - Jha, Girish Nath
A2 - Devi, Sobha Lalitha
A2 - Bali, Kalika
A2 - Ojha, Atul Kr.
PB - European Language Resources Association (ELRA)
Y2 - 20 June 2022
ER -