Multimodal learning for autonomous systems and robotics

Title

Multimodal learning for autonomous systems and robotics

Creator

J., Lekha; Meena Preethi, B.; Yuvaraj, K.P.; Chery, Angelo P.

Description

The realm of autonomous systems and robotics is experiencing a paradigm shift driven by the integration of advanced artificial intelligence (AI) techniques and multimodal learning approaches. This abstract explores the latest advancements and research topics that are propelling the field toward more intelligent, efficient, and versatile autonomous systems. Multimodal learning leverages multiple sensory inputs to enhance the perception and decision-making capabilities of autonomous systems. This involves the integration of visual, auditory, tactile, and other sensory data to form a coherent understanding of the environment. Deep learning techniques, such as multimodal neural networks and crossmodal embeddings, play a pivotal role in this integration, enabling the system to learn joint representations and improve robustness in perception under varying conditions. Computer vision remains a cornerstone of autonomous systems, with advancements in techniques such as real-time object detection, tracking, and high-resolution image synthesis through generative adversarial networks. Vision-based reinforcement learning is also gaining traction, enabling systems to learn from visual inputs and improve their decision-making processes in dynamic environments. The integration of advanced sensors, including high-resolution light detection and ranging, radio detection and ranging, and event-based cameras, enhances the capability of autonomous systems to perceive their surroundings accurately. Multisensor data fusion, using methods like Kalman and particle filters, ensures robust perception even in adverse conditions, providing a comprehensive view of the environment. Innovations in actuation and control systems are fundamental for the development of responsive and adaptive robots. Soft robotics, inspired by biological systems, offers new possibilities in design, modeling, and control. Hybrid control systems facilitate the coordination of multimodal actuation, enhancing the robots versatility and performance. The deployment of high-performance embedded systems, incorporating heterogeneous computing architectures (CPU-GPU-FPGA integration), is vital for real-time data processing and decision-making. Neuromorphic computing and AI hardware accelerators provide low-power solutions that are crucial for the efficiency of autonomous systems. Techniques for uncertainty estimation, outlier detection, and anomaly detection are essential for maintaining system reliability. Advanced robotic perception and cognition, combined with cognitive architectures for autonomous reasoning, enable systems to operate safely in complex and dynamic environments. The interface between humans and robots is evolving, with a focus on multimodal human-robot interaction. Learning from human demonstrations and ensuring safety and trust in human-robot teams are critical areas of research, promoting effective collaboration between humans and robots. Advanced simulation techniques, including high-fidelity physics-based simulations and domain randomization, are employed to test and validate autonomous systems. Virtual reality and augmented reality provide immersive environments for training and testing. Real-time simulation and hardware-in-the-loop testing ensure the robustness and reliability of autonomous systems before deployment. Ethical AI and autonomous decision-making frameworks are being developed to address these issues. Privacy-preserving machine learning techniques and cybersecurity measures are essential for protecting sensitive data and ensuring the security of autonomous systems. This comprehensive overview underscores the rapid advancements and multifaceted nature of multimodal learning and autonomous systems, heralding a new era of intelligent and adaptive robotics capable of transforming numerous industries and improving the quality of human life. 2026 Elsevier Inc. All rights reserved.

Source

Multimodal Learning Using Heterogeneous Data;pp.193-215

Date

01-01-2025

Publisher

Elsevier

Subject

advanced artificial intelligence (AI); Autonomous systems; generative adversarial networks (GANs); Kalman filters; LiDAR; multimodal learning; multimodal neural networks (MMNNs); Multisensor data fusion; robotics

Coverage

J. L., Department of Statistics and Data Science, Christ University, Maharashtra, Pune, India; Meena Preethi B., Department of Software Systems, Sri Krishna Arts and Science College, Tamil Nadu, Coimbatore, India; Yuvaraj K.P., Department of Mechanical Engineering, Sri Krishna College of Engineering and Technology, Tamil Nadu, Coimbatore, India; Chery A.P., Department of Statistics and Data Science, Christ University, Maharashtra, Pune, India

Rights

Restricted Access; Hardcopy may be available in the library

Relation

ISBN: 978-044327528-9; 978-044327529-6;

Format

online

Language

English

Type

Book chapter

Identifier

https://doi.org/10.1016/B978-0-443-27528-9.00005-X

https://www.scopus.com/pages/publications/105032948422?origin=resultslist