A Systematic Review of Challenges, Tools, and Myths of Big Data Ingestion
- Title
- A Systematic Review of Challenges, Tools, and Myths of Big Data Ingestion
- Creator
- Irfan M.; George J.P.
- Description
- Each sector of the digital world generates enormous data as human life continues to transform. Areas like data analytics, data science, knowledge discovery in databases (KDD), machine learning, and artificial intelligence depend on highly distributed data which requires appropriate storage in a data lake. Collecting the data from different heterogeneous sources and creating a single lake of data is called data ingestion. Ironically, data ingestion has been treated as a less important stage in data analysis because it is considered a minor first step. There are several misconceptions in the data and analytics domain about data ingestion. The survey employed in this research presents a list of significant challenges faced by information technology (IT) industries during data ingestion. The available frameworks are compared in terms of standard parameters that are set against the existing challenges and myths. The findings from the comparison are compiled in a tabular format for easy reference. The paper places emphasis on the significance of data ingestion and attempts to present it as a major activity on the big data platform. 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
- Source
- Lecture Notes in Networks and Systems, Vol-462, pp. 481-494.
- Date
- 2022-01-01
- Publisher
- Springer Science and Business Media Deutschland GmbH
- Subject
- Data ingestion; Flume; Ingestion challenges; Ingestion myths; Kafka; NiFi
- Coverage
- Irfan M., CHRIST (Deemed to be University), Lavasa Campus, Bengaluru, India; George J.P., CHRIST (Deemed to be University), Lavasa Campus, Bengaluru, India
- Rights
- Restricted Access
- Relation
- ISSN: 23673370; ISBN: 978-981192210-7
- Format
- Online
- Language
- English
- Type
- Conference paper
Collection
Citation
Irfan M.; George J.P., “A Systematic Review of Challenges, Tools, and Myths of Big Data Ingestion,” CHRIST (Deemed To Be University) Institutional Repository, accessed February 25, 2025, https://archives.christuniversity.in/items/show/20287.