Efficient Multilingual Language Detection Using Machine Learning Algorithms

Title

Efficient Multilingual Language Detection Using Machine Learning Algorithms

Creator

Josemon, Felix; Anil, Anakha; Jayapandian, N.

Description

Natural Language Processing (NLP) is one of the important technologies in recent days, because language detection this NLP is play a vital role. This research focuses on detecting languages using various machine learning algorithms. FastText, Recurrent Neural Networks (RNN), Support Vector Machines (SVM) algorithms are used for this experiment. The following datasets are used to take this result that is Europarl and Tatoeba. The proposed method is to preprocess, train, and test these models. Evaluation is done by measuring precision, recall, and F1 score of the three algorithms. Results show that RNN provides precision close perfect or near-perfect results in both bilingual and multilingual datasets. SVM performs with high precision and recall, but less than RNN. Its performance slightly decreases as the dataset increases. On the other hand, FastText, although fast and efficient, drops significantly in performance as the dataset grows, especially with the inclusion of a third language. It provides an all-inclusive methodology that has pinned the strengths and weaknesses of each algorithm, providing valuable insight into which one best fit real-world language detection task: RNN with their ability to handle complex sequences, SVM for large-scale high-dimensional sparse features, and FastText for simpler, smaller dataset. 2025 IEEE.

Source

2025 World Skills Conference on Universal Data Analytics and Sciences, WorldSUAS 2025;

Date

01-01-2025

Publisher

Institute of Electrical and Electronics Engineers Inc.

Subject

FastText; Long Short-Term Memory; Natural Language Processing; Recurrent Neural Networks; Support Vector Machines

Coverage

Josemon F., Christ University, Department of Computer Science and Engineering, Kengeri Campus, Bangalore, India; Anil A., Christ University, Department of AI, ML and Data Science, Kengeri Campus, Bangalore, India; Jayapandian N., Christ University, Department of Computer Science and Engineering, Kengeri Campus, Bangalore, India

Rights

Restricted Access; Hardcopy may be available in the library

Relation

ISBN: 979-833153925-2;

Format

online

Language

English

Type

Conference paper

Identifier

https://doi.org/10.1109/WorldSUAS66815.2025.11199239

https://www.scopus.com/pages/publications/105022265928?origin=resultslist