ItemArticle

An attention-based loss function and synthetic minority oversampling technique for alleviating class imbalance in predicting diabetes

Title
An attention-based loss function and synthetic minority oversampling technique for alleviating class imbalance in predicting diabetes
Creator
Roy, Santanu; Cherish, Reshma Rachel; Roy, Gifty
Description
Diabetes is a chronic disease due to higher blood sugar (or Glucose) levels in the blood. This study proposes a novel attention-based loss function and a lightweight artificial neural network (ANN) called Diabetic Lite (DB-Lite) for diabetes prediction in the Pima Indian Diabetes Dataset (PIDD). We show that the Pima dataset has many challenges. It is a small and imbalanced dataset; moreover, many features are non-linearly correlated in this dataset. The novelties of this research work are as follows: (i) A novel loss function of attention-based binary cross entropy (ABCE) is proposed for the first time to alleviate the statistical imbalance present within the Pima dataset. This ABCE loss function is incorporated in the DB-Lite model, which is trained from scratch. (ii) A Swish activation function is deployed in the hidden layer of DB-Lite instead of Rectified Linear Unit (ReLU) to deal with the non-linear dependency of features with the final outcome. (iii) The synthetic minority oversampling technique (SMOTE) is used as a pre-processing technique to mitigate the class imbalance problem from the Pima dataset. (iv) An adaptive learning rate is utilized while training the model to speed up the convergence of the DB-Lite model. Our final proposed framework has achieved 99.7% accuracy, 99.4% precision, 99.8% recall, and 99.6% F1 score in testing, which is the best result on this Pima dataset. The Welch t-testing (as a statistical hypothesis testing) and 10-fold cross-validation are utilized to prove the validity of the proposed loss function. 2025
Source
Healthcare Analytics;Volume;7;Issue;;Article No.;100399;
Date
01-01-2025
Publisher
Elsevier Inc.
Subject
Artificial neural network; Attention-based binary cross entropy; Class imbalance; Diabetes prediction; Synthetic minority oversampling technique (SMOTE)
Coverage
Roy S., Pandit Deendayal Energy University (PDEU), Department of Computer Science and Engineering, Gandhinagar, India; Cherish R.R., Ramaiah Institute of Technology, Department of Computer Science and Engineering, Bangalore, India; Roy G., Christ (Deemed to be University), Department of Computer Science and Engineering, Bangalore, India
Rights
All Open Access; Gold Open Access; Green Open Access
Relation
ISSN: 27724425;
Format
online
Language
English
Type
Article

Citation

Roy, Santanu; Cherish, Reshma Rachel; Roy, Gifty, “An attention-based loss function and synthetic minority oversampling technique for alleviating class imbalance in predicting diabetes,” CHRIST (Deemed To Be University) Institutional Repository, accessed June 18, 2026, https://archives.christuniversity.in/items/show/22272.

Output Formats