Evaluation and analysis of capacity scheduler and fair scheduler in hadoop framework on big data technology

Muhammad Salman, Diyanatul Husna, Adhitya Wicaksono, Anak Agung Putri Ratna

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Apache Hadoop is an open source framework that implements MapReduce. It is scalable, reliable, and fault tolerant. Scheduling is an important process in Hadoop MapReduce. It is because scheduling has responsibility to allocate resources for running applications based on resource capacity, queues, running tasks, and the number of users. Changing single node to multi node Hadoop cluster can optimize HDFS, but quite costly. Scheduler performs the function of scheduling based on resource requirements, such as memory, CPU, disk, and network. The most general purpose of scheduling algorithm is minimizing the time of completing a task. Hadoop Scheduling is an independent module where users are able to design their own scheduler based on the application’s actual need. So it can fulfill the specific need of the business in accordance with the desired result. This research will analyze the characteristic of Capacity Scheduler and Fair Scheduler.

Original languageEnglish
Title of host publicationAIVR 2018 - 2018 International Conference on Artificial Intelligence and Virtual Reality
PublisherAssociation for Computing Machinery
Pages1-5
Number of pages5
ISBN (Electronic)9781450366410
DOIs
Publication statusPublished - 23 Nov 2018
Event2018 International Conference on Artificial Intelligence and Virtual Reality, AIVR 2018 - Nagoya, Japan
Duration: 23 Nov 201825 Nov 2018

Publication series

NameACM International Conference Proceeding Series

Conference

Conference2018 International Conference on Artificial Intelligence and Virtual Reality, AIVR 2018
CountryJapan
CityNagoya
Period23/11/1825/11/18

Keywords

  • Big data
  • Capacity scheduler
  • Fair scheduler
  • Hadoop
  • YARN

Fingerprint Dive into the research topics of 'Evaluation and analysis of capacity scheduler and fair scheduler in hadoop framework on big data technology'. Together they form a unique fingerprint.

Cite this