Advance Data Ingestion Framework - Integration, Processing, Transformation, and Loading
- Title
- Advance Data Ingestion Framework - Integration, Processing, Transformation, and Loading
- Creator
- Gupta, Samiksha; Kumar, Ajay; Mishra, Neha; Poonia, Ramesh Chandra
- Description
- The research introduces a new concept known as the Advanced Data Ingestion Framework, which is aimed at enhancing the process of getting into stored information through some intelligent methods like data preprocessing, transformation and loading. By making use of Azure services, the platform considers distributed computing and parallel processing so that structured as well as unstructured data can be incorporated from various origins without any difficulty. To begin with, the proposed framework starts with setting up scalable Azure infrastructure and integrating SAP S4/HANA for secure and efficient data transfer purposes. Within Azure Data Factory the ingestion occurs while Delta Lake ensures proper housekeeping & integrity within the system. It includes creating Power BI dashboards which allow users to see patterns easily and make better decisions based on what they know or can learn. The study brings out the flaws of current data input solutions and emphasizes the urgent requirement for a highly scalable low latency system that can support real time data processing efficiently. It tests the framework under different performance environments showing that it can effectively manage modern data within it. Finally, there is discussion about future improvements such as incorporating more sophisticated analytics or ML models thereby strengthening the decisionmaking process based on available facts. 2025 IEEE.
- Source
- 2025 International Conference on Emerging Trends in Networks and Computer Communications, ETNCC 2025 - Proceedings;pp.160-166
- Date
- 01-01-2025
- Publisher
- Institute of Electrical and Electronics Engineers Inc.
- Coverage
- Gupta S., Jecrc University, Department of Data Analytics, Jaipur, India; Kumar A., Jecrc University, Department of Computer Science & Engineering, Jaipur, India; Mishra N., Jecrc University, Department of Computer Science & Engineering, Jaipur, India; Poonia R.C., Christ University, Department of Computer Science, Bangalore, India
- Rights
- Restricted Access; Hardcopy may be available in the library
- Relation
- ISBN: 979-833152565-1;
- Format
- online
- Language
- English
- Type
- Conference paper
Collection
Citation
Gupta, Samiksha; Kumar, Ajay; Mishra, Neha; Poonia, Ramesh Chandra, “Advance Data Ingestion Framework - Integration, Processing, Transformation, and Loading,” CHRIST (Deemed To Be University) Institutional Repository, accessed June 18, 2026, https://archives.christuniversity.in/items/show/25838.