International Conference on Systems, Science, Control, Communication, Engineering and Technology 2015


ICSSCCET 2015 Kokula Krishna Hari K
Publication Meta Value
Short Title ICSSCCET 2015
Publisher ASDF, India
ISBN 13 978-81-929866-1-6
ISBN 10 81-929866-1-6
Language English
Type Hard Bound - Printed Book
Copyrights ICSSCCET Organizers / DCRC, London, UK
Editor-in-Chief Ramachandran T
Conference Dates 10 - 11, August 2015
Venue Country Karpagam Institute of Technology, Coimbatore, India
Submitted Papers 410
Acceptance Rate 4.11%
Website www.icssccet.org

Paper 017

Big Data - Reduced Task Scheduling

Big Data - Reduced Task Scheduling

Kokula Krishna Hari K1, Vignesh R2, Long CAI3, Rajkumar Sugumaran4

1Chief Scientist, Techno Forum Research and Development Center, Hong Kong
2Life Member, Association of Scientists, Developers and Faculties, India
3University of Hong Kong, HKSAR , Hong Kong
4Vice-President (HR), Techno Forum Research and Development Center, Bangkok


Inspired by the success of Apache’s Hadoop this paper suggests an innovative reduce task scheduler. Hadoop is an open source implementation of Google’s MapReduce framework. Programs which are written in this functional style are automatically executed and parallelized on a large cluster of commodity machines. The details how to partition the input data, setting up the program's for execution across a set of machines, handling machine failures and managing the required inter-device communication is taken care by runtime system. In existing versions of Hadoop, the scheduling of map tasks is done with respect to the locality of their inputs in order to diminish network traffic and improve performance. On the other hand, scheduling of reduce tasks is done without considering data locality leading to degradation of performance at requesting nodes. In this paper, we exploit data locality that is inherent with reduce tasks. To accomplish the same, we schedule them on nodes that will result in minimum data- local traffic. Experimental results indicate an 11-80 percent reduction in the number of bytes shuffled in a Hadoop cluster.

Author's Profile

Kokula Krishna Hari K : Profile

Vignesh R : Profile

Long CAI : Profile

Rajkumar Sugumaran : Profile

Cite this Article as Follows

Kokula Krishna Hari K, Vignesh R, Long CAI, Rajkumar Sugumaran."Big Data - Reduced Task Scheduling." International Conference on Systems, Science, Control, Communication, Engineering and Technology (2015): 79-84. Print.