Machine Translation from Tulu to English using Transformer Architecture

The Machine Translation of Tulu to English is performed using the Transformer architecture. The Recurrent structures will be replaced by self attention, which aids in our comprehension of the relationship between the words and their context within the sentence. The limited parallelism, slow training, and difficulties in capturing long-range dependencies of current machine translation techniques based on Recurrent Neural Networks (RNNs), like GRU and LSTM architectures, lead to higher computational costs and reduced translation accuracy. Large datasets require longer training times because these models process input sequences sequentially, which restricts them from fully utilizing GPU-based parallel computation. Additionally, learning contextual relationships across long phrases is adversely affected by the vanishing and exploding gradient issues present in recurrent architectures. The Transformer cuts the time needed for the accuracy by using sinusoidal positional encoding and multiple heads of focus to observe the whole chain through computation in parallel as opposed to the conventional RNN-inspired models of Seq2Seq that rely upon on GRU or LSTM units for computation. 9,464 Tulu-English sentence pairs which have been tokenized and padded for successful acquisition were utilized as the dual language dataset on which the model was trained. Training employed a greedy decoding mechanism during prediction and developed a custom loss function alongside masking to deal with inputs and outputs of variable lengths. Standardized translation metrics like BLEU score, loss visualizations, and training and validation accuracy were employed to evaluate the system. The Transformer works efficiently for low-resource languages like Tulu, as proven by experimental results indicating greater translation quality with a tenfold reduction in the computational power required to train the model.