Bridging Signs to Sentences: Enhancing Sign Language Interpretation

Title

Bridging Signs to Sentences: Enhancing Sign Language Interpretation

Creator

Sukruth, S.; Nickolas, S.; Jayapriya, J.; Vinay, M.

Description

This paper mainly focuses on bridging the gap between sign language recognition (SLR) and sentence formation by integrating the recognition of different signs or machine learning models with large language models (LLMs), which results in contributing and enhancing the communication of the deaf community. There are around 466million deaf individuals worldwide, where they primarily rely on Sign language for communication. Current SLR technologies have certain limitations that deal with difficulty in sentence formation and high processing requirements. This papers dataset consists of 36 classes, where 26 of them are alphabets, and the rest 9 are numbers, each consisting of 500 images. Therefore, a total of 18,000 images are present in the dataset for accurate prediction of sign languages. MediaPipe, developed by Google, is used as a tool for feature extraction by identifying 21 hand landmarks. The features extracted are then passed onto an ML model (like a Random Forest), and then the result of this model is passed on to an LLM (here, Groqs Gemma-7b-it) that forms a sentence based on the predictions. The ML model achieved a high accuracy of 99.54% and the LLMs achieved an accuracy of 93.75%. With the combination of sign language recognition and advanced modeling, this work helps to bridge signs to sentences, providing to the deaf community. The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2026.

Source

Lecture Notes in Networks and Systems;Volume;1542 LNNS;pp.39-52

Date

01-01-2026

Publisher

Springer Science and Business Media Deutschland GmbH

Subject

Image feature extraction; Large language model (LLM); MediaPipe hand tracking; Real-time sign language translation; Sentence formation from sign language; Sign language recognition (SLR)

Coverage

Sukruth S., CHRIST Deemed to be University, Bengaluru, India; Nickolas S., National Institute of Technology, Tiruchirappalli, India; Jayapriya J., CHRIST Deemed to be University, Bengaluru, India; Vinay M., CHRIST Deemed to be University, Bengaluru, India

Rights

Restricted Access; Hardcopy may be available in the library

Relation

ISSN: 23673370; ISBN: 978-981969681-9;

Format

online

Language

English

Type

Conference paper

Identifier

https://doi.org/10.1007/978-981-96-9682-6_4

https://www.scopus.com/pages/publications/105028319078?origin=resultslist