Hyper-Parameter Tuning based on Random Search for DenseNet Optimization

Ari Nugroho, Heru Suhartanto

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Deep learning is a machine learning technology that is currently experiencing rapid development. One of the deep learning architecture models is Densely Connected Convolutional Networks (DenseNet), which connects each layer along with feature maps to all next layers. In other words, the next layer receives input feature maps from all previous layers. Because of this density architecture, the training time is longer and requires much memory. One of the ways to overcome these problems is to optimize using the batching strategy method. This batching method uses adaptive batch size with the ratio of learning rate during training to get faster training time without reducing accuracy. To obtain the optimal performance of this batching, a proper selection of hyper-parameter learning rate and batch size is needed. In this study, we perform tuning this hyper-parameter to get the optimal value. The selection of hyper-parameters uses a random search to select candidates for learning rate and batch size, and then an experiment is conducted on the candidates. Random search randomly selects the hyper-parameters, then training and scoring to the hyper-parameters. Our experiment results show that the lower limit of the batch size used is 64, while the optimal learning rate range is in the range of 0.1 - 0.3. The average accuracy obtained from using this hyper-parameter is 95%. Our experiments run on GPU RTX 2080 Ti with CIFAR-10 and CIFAR-100 datasets.

Original languageEnglish
Title of host publication7th International Conference on Information Technology, Computer, and Electrical Engineering, ICITACEE 2020 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages96-99
Number of pages4
ISBN (Electronic)9781728172255
DOIs
Publication statusPublished - 24 Sep 2020
Event7th International Conference on Information Technology, Computer, and Electrical Engineering, ICITACEE 2020 - Semarang, Indonesia
Duration: 24 Sep 202025 Sep 2020

Publication series

Name7th International Conference on Information Technology, Computer, and Electrical Engineering, ICITACEE 2020 - Proceedings

Conference

Conference7th International Conference on Information Technology, Computer, and Electrical Engineering, ICITACEE 2020
CountryIndonesia
CitySemarang
Period24/09/2025/09/20

Keywords

  • batching strategy
  • DenseNet
  • hyper-parameter
  • random search

Fingerprint Dive into the research topics of 'Hyper-Parameter Tuning based on Random Search for DenseNet Optimization'. Together they form a unique fingerprint.

Cite this