Bigdata Analytics With Spark

SUBHASH KUMAR¹

¹IT Department, St Xavier's College, Mahapalika Marg, New Marine Lines, Mumbai, PIN-400001, Maharashtra, India

Abstract

Current generation is witnessing data explosion most of it is unstructured and is called Big Data. This data has characteristics of high volume, velocity, variety and veracity. HDFS, GFS, Ceph, Lustre, PVFS etc are used as file system for storing Big Data. MapReduce processes program in parallel across clusters and generates output. Spark framework improves performance by 10x when datasets are stored in hard disk and performance improves by 100x when data is stored in memory. This paper proposes optimization of Big Data processing using Spark framework.

Keywords

• Volume, velocity, veracity, Big Data, HDFS, Spark

Author's Profile

Author profile can be generated and linked through our partners World Book of Researchers. To include your profile online Click Here. After it is approved, please email to edlib @ asdf.res.in to create a link with all the papers.

• SUBHASH KUMAR : Profile

e-AID

Cite this Article as Follows

SUBHASH KUMAR. "Bigdata Analytics With Spark." International Conference on Information Engineering, Management and Security (2015): 129-133. Print.