Bridging Traditional NLP and Deep Learning: Comparative Study on Text Categorization Performance
subject: DistilBERT; LoRA; Machine Learning; NLP; PEFT; Semantic Embeddings; Text Classification; Transformer Models
description: Text categorization is an important area of Natural Language Processing (NLP) that is used to automatically organize textual information into a set of specific categories. This study is a comparative study of models that use statistical features and models that use transformers, using the example of DistilBERT-base-uncased fine-tuned and LoRA (Parameter-Efficient Fine-Tuning, PEFT). The data extracted on the Kaggle site is presented in the form of labeled text samples of five classes Business, Entertainment, Sport, Tech, and Politics. Conventional models such as Logistic Regression, Random Forest and XGBoost were trained on manually crafted word-level features (word count, mean word length and punctuation ratio) and had precisions up to 94.7%. Comparatively, the given DistilBERT-LoRA model used semantic embeddings to find the contextual dependencies and managed to reach the total accuracy of 97, precision of 97, and the recall of 96. The training and validation loss curves showed the stable convergence without overfitting, and the confusion matrix showed the consistent performance at all the classes with minimum misclassification. Comparative analysis indicated that semantic embeddings are much better than statistical models because they enhance contextual perception and strength of classification. The findings confirm the effectiveness and scalability of the LoRA-based fine-tuning, which offers an efficient but lightweight strategy in the context of real-world settings to achieve high-performance text categorization. 2025 IEEE.
creator: Srinivas, T. Aditya Sai; Shivajyothi, Aenugu; Pal Pandian, P.; Nandini, M. Raja; Periyasamy, Rajeswari; Kaliappan, S.
source: 2025 IEEE 3rd Global Conference on Wireless Computing and Networking, GCWCN 2025;
publisher: Institute of Electrical and Electronics Engineers Inc.
date: 01-01-2025
rights: Restricted Access; Hardcopy may be available in the library
relation: ISBN: 979-833150335-2;
language: English
type: Conference paper
coverage: Srinivas T.A.S., Ravindra College of Engineering for Women, Department of Computer Science and Engineering, Andhra Pradesh, Kurnool, 518452, India; Shivajyothi A., Geethanjali College of Engineering and Technology, Department of Computer Science and Engineering, Telangana, Medchal, 501301, India; Pal Pandian P., Christ University, Robotics and Mechatronics, School of Engineering and Technology, Bangalore, India; Nandini M.R., Mallareddy Engineering College for Women, Dept of CSE, Telangana, Hyderabad, 500100, India; Periyasamy R., Dayananda Sagar College of Engineering, Department of Electronics and Telecommunication Engineering, Karnataka, Bangalore, 560078, India; Kaliappan S., Lovely Professional University, Division of Research and Development, Punjab, Phagwara, 144411, India