Online marginalized linear stacked denoising autoencoders for learning from big data stream

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Citations (Scopus)

Abstract

Big non-stationary data, which comes in gradual fashion or stream, is one important issue in the application of big data to train deep learning machines. In this paper, we focused on a unique variant of traditional autoencoder, which is called Marginalized Linear Stacked Denoising Autoencoder (MLSDA). MLSDA uses a simple linear model. It is faster and uses less number of parameters than the traditional SDA. It also takes advantages of convex optimization. It has better improvement in the bag of words feature representation. However, the traditional SDA with stochastic gradient descent has been more widely accepted in many applications. The stochastic gradient descent is naturally an online learning. It makes the traditional SDA more scalable for streaming big data. This paper proposes a simple modification of MLSDA. Our modification uses matrix multiplication concept for online learning. The experiment result showed the similar accuracy level compared with a batch version of MLSDA and using lower computation resources. The online MLSDA will improve the scalability of MLSDA for handling streaming big data that representing bag of words features for natural language processing, information retrieval, and computer vision.

Original languageEnglish
Title of host publicationICACSIS 2015 - 2015 International Conference on Advanced Computer Science and Information Systems, Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages227-235
Number of pages9
ISBN (Electronic)9781509003624
DOIs
Publication statusPublished - 19 Feb 2016
EventInternational Conference on Advanced Computer Science and Information Systems, ICACSIS 2015 - Depok, Indonesia
Duration: 10 Oct 201511 Oct 2015

Publication series

NameICACSIS 2015 - 2015 International Conference on Advanced Computer Science and Information Systems, Proceedings

Conference

ConferenceInternational Conference on Advanced Computer Science and Information Systems, ICACSIS 2015
Country/TerritoryIndonesia
CityDepok
Period10/10/1511/10/15

Keywords

  • Autoencoder
  • big data
  • deep learning
  • denoising
  • online
  • Sequential

Fingerprint

Dive into the research topics of 'Online marginalized linear stacked denoising autoencoders for learning from big data stream'. Together they form a unique fingerprint.

Cite this