Big sensor-generated data streaming using Kafka and Impala for data storage in Wireless Sensor Network for CO2 monitoring

Rindra Wiska, Novian Habibie, Ari Wibisono, Widijanto Satyo Nugroho, Petrus Mursanto

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

Wireless Sensor Network (WSN) is a system that have a capability to conduct data acquisition and monitoring in a wide sampling area for a long time. However, because of its big-scale monitoring, amount of data accumulated from WSN is very huge. Conventional database system may not be able to handle its big amount of data. To overcome that, big data approach is used for an alternative data storage system and data analysis process. This research developed a WSN system for CO2 monitoring using Kafka and Impala to distribute a huge amount of data. Sensor nodes gather data and accumulated in temporary storage then streamed via Kafka platform to be stored into Impala database. System tested with data gathered from our-own made sensor nodes and give a good performance.

Original languageEnglish
Title of host publication2016 International Workshop on Big Data and Information Security, IWBIS 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages97-101
Number of pages5
ISBN (Electronic)9781509034772
DOIs
Publication statusPublished - 6 Mar 2017
Event2016 International Workshop on Big Data and Information Security, IWBIS 2016 - Jakarta, Indonesia
Duration: 18 Oct 201619 Oct 2016

Publication series

Name2016 International Workshop on Big Data and Information Security, IWBIS 2016

Conference

Conference2016 International Workshop on Big Data and Information Security, IWBIS 2016
CountryIndonesia
CityJakarta
Period18/10/1619/10/16

Fingerprint Dive into the research topics of 'Big sensor-generated data streaming using Kafka and Impala for data storage in Wireless Sensor Network for CO<sub>2</sub> monitoring'. Together they form a unique fingerprint.

  • Cite this

    Wiska, R., Habibie, N., Wibisono, A., Nugroho, W. S., & Mursanto, P. (2017). Big sensor-generated data streaming using Kafka and Impala for data storage in Wireless Sensor Network for CO2 monitoring. In 2016 International Workshop on Big Data and Information Security, IWBIS 2016 (pp. 97-101). [7872896] (2016 International Workshop on Big Data and Information Security, IWBIS 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IWBIS.2016.7872896