Visual-audio fusion in multimedia content analysis

Title

Visual-audio fusion in multimedia content analysis

Creator

Gnanamonickam, A. Arul Selvan; M., Savithri

Description

The analysis of multimedia material has grown in importance due to the rapid expansion of digital media. Exploiting the combination of auditory and visual modalities for improved comprehension and interpretation of multimedia material has gained popularity in recent years. An extensive review of the methods, strategies, and uses of visual-audio fusion in multimedia content analysis is provided in this chapter. Combining auditory and visual modalities provides a number of benefits over single-modal analysis, such as increased robustness, deeper semantic comprehension, and better user experience. This chapter investigates a variety of fusion approaches, from late combination at the decision near to early synthesis at the feature nearby. Besides, it studies sophisticated fusion methods that allow for efficient integraton of data across modalities, such cross-modal attention processes and multimodal deep learning. Also looks at several other areas, such as multimedia retrieval, event detection, sentiment analysis, and emotional computing, where visual-audio fusion has been used successfully. It dialogs about the difficulties and potential paths ahead for the area, including how to deal with modality inconsistencies, manage massive multimedia information, and create fusion models that are understandable. To sum up, visual-audio fusion presents new possibilities for comprehending and analyzing complicated multimedia data and has the potential to significantly advance multimedia content analysis. 2026 Elsevier Inc. All rights reserved.

Source

Multimodal Learning Using Heterogeneous Data;pp.177-191

Date

01-01-2025

Publisher

Elsevier

Subject

cross-model; Fusion; multimedia; multimedia analysis; sophisticated fusion; visual-audio

Coverage

Gnanamonickam A.A.S., Department of Software System and Computer Science (PG), KG College of Arts and Science, Tamil Nadu, Coimbatore, India; M. S., Department of Data Science, Christ University, Karnataka, Bengaluru, India

Rights

Restricted Access; Hardcopy may be available in the library

Relation

ISBN: 978-044327528-9; 978-044327529-6;

Format

online

Language

English

Type

Book chapter

Identifier

https://doi.org/10.1016/B978-0-443-27528-9.00008-5

https://www.scopus.com/pages/publications/105032931832?origin=resultslist