Sketch plus colorization deep convolutional neural networks for photos generation from sketches

Vinnia Kemala Putri, Mohamad Ivan Fanany

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Citations (Scopus)

Abstract

In this paper, we introduce a method to generate photos from sketches using Deep Convolutional Neural Networks (DCNN). This research proposes a method by combining a network to invert sketches into photos (sketch inversion net) with a network to predict color given grayscale images (colorization net). By using this method, the quality of generated photos is expected to be more similar to the actual photos. We first artificially constructed uncontrolled conditions for the dataset. The dataset, which consists of hand-drawn sketches and their corresponding photos, were pre-processed using several data augmentation techniques to train the models in addressing the issues of rotation, scaling, shape, noise, and positioning. Validation was measured using two types of similarity measurements: pixel-difference based and human visual system (HVS) which mimics human perception in evaluating the quality of an image. The pixel-difference based metric consists of Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR) while the HVS consists of Universal Image Quality Index (UIQI) and Structural Similarity (SSIM). Our method gives the best quality of generated photos for all measures (844.04 for MSE, 19.06 for PSNR, 0.47 for UIQI, and 0.66 for SSIM).

Original languageEnglish
Title of host publicationProceedings - 2017 4th International Conference on Electrical Engineering, Computer Science and Informatics, EECSI 2017
EditorsHatib Rahmawan, Mochammad Facta, Munawar A. Riyadi, Deris Stiawan
PublisherInstitute of Advanced Engineering and Science
ISBN (Electronic)9781538605486
DOIs
Publication statusPublished - 22 Dec 2017
Event4th International Conference on Electrical Engineering, Computer Science and Informatics, EECSI 2017 - Yogyakarta, Indonesia
Duration: 19 Sep 201721 Sep 2017

Publication series

NameInternational Conference on Electrical Engineering, Computer Science and Informatics (EECSI)
Volume2017-December
ISSN (Print)2407-439X

Conference

Conference4th International Conference on Electrical Engineering, Computer Science and Informatics, EECSI 2017
Country/TerritoryIndonesia
CityYogyakarta
Period19/09/1721/09/17

Keywords

  • Colorization
  • Deep convolutional neural networks
  • Sketch inversion

Fingerprint

Dive into the research topics of 'Sketch plus colorization deep convolutional neural networks for photos generation from sketches'. Together they form a unique fingerprint.

Cite this