Deep learning architectures for multimodal fusion

Title

Deep learning architectures for multimodal fusion

Creator

Savithri, M.; Arul Selvan Gnanamonickam, A.

Description

The advancement in technology during the recent years has provided deep learning technology as an emerging and powerful paradigm which can be used for processing and understanding complex data across various domains. Multimodal fusion is integrating the information that is collected from various sources or modalities, which requires a comprehensive understanding of data like autonomous driving, medical diagnosis, etc. In this chapter, we will explore the various advanced deep learning architectures that have been specially designed based on the multimodal fusion. The various challenges that are being faced in multimodal data, which include heterogeneity, noise, reliability of data, etc. Various deep learning architectures that are built to address the various challenges, like convolutional neural networks, recurrent neural networks, are reviewed, and the suitability of the fusion strategies is highlighted. The various techniques that are used for combining the information from disparate modalities, like early fusion, late fusion, and hybrid approaches, are also discussed with their pros and cons. Various real-time applications in the field of healthcare, multimedia, robotics, etc., are demonstrated based on the impact of the architectures. Finally, the potential of deep learning architecture based on the revolutionary multimodal fusion will be discussed. 2026 Elsevier Inc. All rights reserved.

Source

Multimodal Learning Using Heterogeneous Data;pp.57-68

Date

01-01-2025

Publisher

Elsevier

Subject

convolutional neural networks; deep learning; explainable AI (XAI); Multimodal fusion; multimodal generative adversarial networks; recurrent neural networks

Coverage

Savithri M., Department of Data Science, Christ University, Karnataka, Bengaluru, India; Arul Selvan Gnanamonickam A., Department of Software System and Computer Science (PG), KG College of Arts and Scienc, Tamil Nadu, Coimbatore, India

Rights

Restricted Access; Hardcopy may be available in the library

Relation

ISBN: 978-044327528-9; 978-044327529-6;

Format

online

Language

English

Type

Book chapter

Identifier

https://doi.org/10.1016/B978-0-443-27528-9.00004-8

https://www.scopus.com/pages/publications/105032924333?origin=resultslist