Verification and validation of MapReduce program model for parallel K-means algorithm on Hadoop cluster

Title

Verification and validation of MapReduce program model for parallel K-means algorithm on Hadoop cluster

Creator

Kumar A.; Kiran M.; Prathap B.R.

Description

With the development of information technology, a large volume of data is growing and getting stored electronically. Thus, the data volumes processing by many applications will routinely cross the petabyte threshold range, in that case it would increase the computational requirements. Efficient processing algorithms and implementation techniques are the key in meeting the scalability and performance requirements in such scientific data analyses. So for the same here, we have p analyzed the various MapReduce Programs and a parallel clustering algorithm (PKMeans) on Hadoop cluster, using the Concept of MapReduce. Here, in this experiment we have verified and validated various MapReduce applications like wordcount, grep, terasort and parallel K-Means Clustering Algorithm. We have found that as the number of nodes increases the execution time decreases, but also some of the interesting cases has been found during the experiment and recorded the various performance change and drawn different performance graphs. This experiment is basically a research study of above MapReduce applications and also to verify and validate the MapReduce Program model for Parallel K-Means algorithm on Hadoop Cluster having four nodes. 2013 IEEE.

Source

2013 4th International Conference on Computing, Communications and Networking Technologies, ICCCNT 2013

Date

2013-01-01

Subject

Grep; Hadoop; K-means; Machine learning; MapReduce; Terasort; Wordcount

Coverage

Kumar A., Department of Computer Science and Engineering, Christ University, Faculty of Engineering Bangalore, Karnataka, India; Kiran M., Department of Computer Science and Engineering, Christ University, Faculty of Engineering Bangalore, Karnataka, India; Prathap B.R., Department of Computer Science and Engineering, Christ University, Faculty of Engineering Bangalore, Karnataka, India

Rights

All Open Access; Green Open Access

Relation

ISBN: 978-147993926-8

Format

Online

Language

English

Type

Conference paper

Identifier

https://doi.org/10.1109/ICCCNT.2013.6726852

https://www.scopus.com/inward/record.uri?eid=2-s2.0-84894488015&doi=10.1109%2fICCCNT.2013.6726852&partnerID=40&md5=bc9ec99c84e42a1d59e93b69c36233ed