Performance Evaluation of Alternative file systems over HDFS

Udhayakumar Shanmugam¹, Dhinakaran K², Silviyanancy J³, Nikhiljacob⁴

¹B.S.Abdur Rahman University
^2,3Saveetha University

Abstract

In the last decade or so there has been an increased use and growth of social media, unconventional web technologies, computers and mobile applications, which have all encouraged development of various database models. Recent datasets are extremely costly and unpractical to administer with SQL databases due to lack of structure, high scalability, and elasticity that is needed. The No SQL data stores such as Mongo DB and Cassandra provide a desirable platform for fast and efficient data queries. With the introduction of the “Big Data” the size and structure of data have become highly dynamic and complex. This paper exhibits evaluation of the Cassandra a No SQL database when used in conjunction with the Hadoop Map Reduce engine, also the Ceph File system developed by Data Stax and the Lustre File system which all can be used as an alternative to HDFS (Hadoop Distributed File System). We provide a brief overview of MapReduce and Hadoop and then show some of the shortcomings of Hadoop + HDFS. Ceph maximizes the separation between data and metadata management by replacing allocation tables with a pseudo-random data distribution function. Also we have evaluated theoretical and actual performance of Lustre and HDFS for a variety of workloads in both traditional and Map/Reduce-based applications.

Keywords

• Hadoop, Bigdata, Ceph file system, HDFS. Cassandra, luster file system

Author's Profile

Author profile can be generated and linked through our partners World Book of Researchers. To include your profile online Click Here. After it is approved, please email to edlib @ asdf.res.in to create a link with all the papers.

• Udhayakumar Shanmugam : Profile
• Dhinakaran K : Profile
• Silviyanancy J : Profile
• Nikhiljacob : Profile

e-AID

Cite this Article as Follows

Udhayakumar Shanmugam, Dhinakaran K, Silviyanancy J, Nikhiljacob. Performance Evaluation of Alternative file systems over HDFS. International Conference on Inter Disciplinary Research in Engineering and Technology 2015. Vol. 1. Chennai: Association of Scientists, Developers and Faculties, 2015. 225-234. Print.