A Novel Approach for Machine Reading Comprehension using BERT-based Large Language Models
subject: ALBERT; DistilBERT; Longformer; Machine Reading Comprehension; Span Extraction; Transformers
description: Teaching machines to learn the information from the natural language documents remains an arduous task because it involves natural language understanding of contexts, excerpting the meaningful insights, and deliver the answer to the questions. Machine Reading Comprehension (MRC) tasks can identify and understand the content from the natural language documents by asking the model to answer questions. The model can extract the answer from the given context or other external repositories. This article proposes a novel Context Minimization method for MRC Span Extraction tasks to improve the accuracy of the models. The Context Minimization method constitutes two subprocedures, Context Reduction and Sentence Aggregation. The proposed model reduced the context with the most relevant sequences for answering by estimating the sentence embedding between the question and the sequences involved in the context. The Context Reduction facilitates the model to retrieve the answer efficiently from the minimal context. The Sentence Aggregation improves the quality of answers by aggregating the most relevant shreds of evidence from the context. Both methods have been developed from the two notable observations from the empirical analysis of existing models. First, the models with minimal context with efficient masking can improve the accuracy and the second is the issue with the scatted sequences on the context that may lead to partial or incomplete answering. The Context Minimization method with Fine-Tuned BERT model compared with the ALBERT, DistilBERT, and Longformer models and the experiments with these models have shown significant improvement in results. 2024 IEEE.
creator: Varghese N.; Shereef S.; Joy H.K.; Ramasamy G.; Sridevi R.; Cynthia T.; Rajeshkanna R.
source: Proceedings of CONECCT 2024 - 10th IEEE International Conference on Electronics, Computing and Communication Technologies
publisher: Institute of Electrical and Electronics Engineers Inc.
date: 2024-01-01
rights: Restricted Access
relation: ISBN: 979-835038592-2
format: Online
language: English
type: Conference paper
coverage: Varghese N., Christ University, Department of Computer Science, Karnataka, Bangalore, 560029, India; Shereef S., Jain (Deemed to Be University), Department of Cs and It, Karnataka, Bangalore, 560059, India; Joy H.K., Christ University, Department of Computer Science, Karnataka, Bangalore, 560029, India; Ramasamy G., Christ University, Department of Computer Science, Karnataka, Bangalore, 560029, India; Sridevi R., Christ University, Department of Computer Science, Karnataka, Bangalore, 560029, India; Cynthia T., Christ University, Department of Computer Science, Karnataka, Bangalore, 560029, India; Rajeshkanna R., Christ University, Department of Computer Science, Karnataka, Bangalore, 560029, India