ItemConference Paper

Data Ingestion - Cloud based Ingestion Analysis using NiFi

Title
Data Ingestion - Cloud based Ingestion Analysis using NiFi
Creator
Irfan M.; Reena; George J.
Description
Data Ingestion has been an integral part of Data Analysis. Bringing the data from various heterogeneous sources to one common place and ensuring the data is captured in the appropriate format is the key for performing any Big data task. Data ingestion is performed using multiple frameworks across the industry and they all have their own set of benefits and drawbacks. Apache NiFi is one popular ingestion framework which is used widely and does Ingestion effectively. Ingestion is performed on various sources and the data is generally stored in clusters or cloud storage. In this paper, we have done the File Data Ingestion using the NiFi framework on a local machine and then on two cloud-based platforms, namely Google Cloud Platform (GCP) and Amazon Web Services (AWS). The objective is to understand the latency and performance of the NiFi tool on Cloud-based Ingestion and provide a comparative study against the typical Data Ingestion. The entire setup was done on a local machine and two corresponding cloud platforms namely GCP and AWS. The findings from the comparative analysis have been compiled in a tabular format and graphs are created for easy reference. The paper places emphasis on the significance of NiFi's data ingestion performance on Cloud Platform and attempts to present it as a major activity on the data ingestion platform for Cloud Ingestion Solution. 2023 IEEE.
Source
International Conference on Self Sustainable Artificial Intelligence Systems, ICSSAS 2023 - Proceedings, pp. 1099-1107.
Date
2023-01-01
Publisher
Institute of Electrical and Electronics Engineers Inc.
Subject
Cloud Ingestion; Data Ingestion; File Ingestion; Ingestion Analysis using NiFi; Ingestion Challenges; NiFi Architecture
Coverage
Irfan M., Christ (Deemed to Be University), Bangalore, India; Reena, Christ (Deemed to Be University), Bangalore, India; George J., Christ (Deemed to Be University), Bangalore, India
Rights
Restricted Access
Relation
ISBN: 979-835030085-7
Format
Online
Language
English
Type
Conference paper

Collection

Conference Papers

Citation

Irfan M.; Reena; George J., “Data Ingestion - Cloud based Ingestion Analysis using NiFi,” CHRIST (Deemed To Be University) Institutional Repository, accessed February 24, 2025, https://archives.christuniversity.in/items/show/19760.

Output Formats