Introduction to multimodal learning and heterogenous data

Title

Introduction to multimodal learning and heterogenous data

Creator

Ramasamy, Gunavathi; Patkar, Kalpesh Dilip; Chery, Angelo P.

Description

With the rising advancements in computation, technology, and many innovative evolutions coming into play, multimodal learning is one of the most rapidly growing fields within the domain of artificial intelligence and Machine Learning. It mainly focuses on integrating information from multiple sources called "modalities," allowing the systems to utilize the varieties of data to furtherenhance their understanding and performance. These so-called modalities make use of various types of data in the form of text, images, audio, and sensor readings. They are able to process complex information due to these modalities and thus provide more insightful results for the tasks that they are assigned. Another important aspect of multimodal learning is heterogeneous datadata that differs significantly in structure, format, and origin. This type of data falls mainly into three categories, comprising structured data, which is quite organized and therefore easy to locate or search, as in the case of the database records. Then comes unstructured data characterized by the free form, which comprises mainly social media posts, videos, and images. In addition, it has been possible to separate semistructured data. It incorporates some features of being ordered, like the metadata included in XML or JSON files; however, a fixed schema does not apply. The understanding of the kind is important because each type calls for a different problem, and each type poses new opportunities in analysis. Handling the heterogeneous data effectively is all the more important because the said system will be fed heterogeneous data, and if its combination and analysis go reasonably logically, it is expected to be a source for multimodal systems. The ability to merge structured, unstructured, and semistructured data improves performance across a wide range of tasks, including but not limited to common applications like image recognition, sentiment analysis, and decision-making processes in autonomous systems. For example, in the multimodal learning case, it would be beneficial for the system that learns customer feedback to merge textual reviews, audio recordings of customer interactions, and visual data from product images. It has been known to yield a much clearer picture of what customers really want and how they actually behave. This chapter introduces notions of multimodal learning as well as heterogeneous datatheir characteristics, types, sources, and practical usage. It will attempt to establish a basic understanding of these two concepts in relation to each other in order to support more advanced applications through machine learning. In a review of the possible compositions between multimodal learning as well as heterogeneous data, the chapter will introduce their importance regarding the creation of intelligent systems that can address complex, intricate tasks across differing fields. As we enter the data age, with multiple sources churning out data at unprecedented rates that appear to have no bounds, integration of multimodal learning with heterogeneous data cannot be ignored. This will be vital for coming up with flexible yet useful applications to real-world problems. This region is promising for systems that perceive, interpret, and respond to the variability of information in a fashion similar to human reasoning and decision-making. Future application of artificial intelligence in the life of man will result from continuous research in the areas of multimodal learning and heterogeneous data. 2026 Elsevier Inc. All rights reserved.

Source

Multimodal Learning Using Heterogeneous Data;pp.1-15

Date

01-01-2025

Publisher

Elsevier

Subject

artificial intelligence; data science; heterogeneous data; knowledge management; Machine learning; multimodal learning

Coverage

Ramasamy G., School of Sciences, Christ University, Karnataka, Bengaluru, India; Patkar K.D., School of Law, CHRIST University, Karnataka, Bengaluru, India; Chery A.P., School of Sciences, Christ University, Karnataka, Bengaluru, India

Rights

Restricted Access; Hardcopy may be available in the library

Relation

ISBN: 978-044327528-9; 978-044327529-6;

Format

online

Language

English

Type

Book chapter

Identifier

https://doi.org/10.1016/B978-0-443-27528-9.00002-4

https://www.scopus.com/pages/publications/105032942844?origin=resultslist