Emotion-Aware Sign Language Recognition Using CNN and UNET Architectures
subject: Convolutional Neural Network (CNN); Deep Learning; Facial Emotion Recognition (FER); Human-Computer Interaction (HCI); Sign Language Recognition (SLR); UNET
description: This paper proposes an AI-based system for the recognition of sign language with the detection of emotions for more expressive communication among speech-impaired and hearing individuals and others. Traditional sign language systems focus mainly on the aspect of hand gestures and neglect the signs for emotions that add meaning and context. In order to overcome the limitation, the project proposes a system that utilizes Convolutional Neural Nets (CNNs) for the recognition of hand signs and UNET for the segmentation of the picture so that the area of the hands can be discriminated from the background. Facial Emotion Recognition (FER) is also incorporated in order to detect signs such as happiness, sadness, or anger. Overall, the parts together constitute a multimodal recognition system that can read signs and emotions and produce more natural and expressive outputs. The paper delves into architecture, dataset challenges, and implementation concepts with publicly available databases such as RWTH-PHOENIX-Weather 2014T. The combined approach can enhance inclusivity and access in learning, communication, and assistive technology. 2025 IEEE.
creator: Thomas, Aksa Maria; Kirubanand, V.B.
source: Proceedings of the 2025 International Conference on Computational Innovations and Sustainable Technologies, ICCIST 2025;
publisher: Institute of Electrical and Electronics Engineers Inc.
date: 01-01-2025
rights: Restricted Access; Hardcopy may be available in the library
relation: ISBN: 979-833159676-7;
language: English
type: Conference paper
coverage: Thomas A.M., CHRIST (Deemed to be University), Department of Computer Science, India; Kirubanand V.B., CHRIST (Deemed to be University), Department of Computer Science, India