Sketch plus colorization deep convolutional neural networks for photos generation from sketches

Vinnia Kemala Putri, Mohamad Ivan Fanany

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)

Abstract

In this paper, we introduce a method to generate photos from sketches using Deep Convolutional Neural Networks (DCNN). This research proposes a method by combining a network to invert sketches into photos (sketch inversion net) with a network to predict color given grayscale images (colorization net). By using this method, the quality of generated photos is expected to be more similar to the actual photos. We first artificially constructed uncontrolled conditions for the dataset. The dataset, which consists of hand-drawn sketches and their corresponding photos, were pre-processed using several data augmentation techniques to train the models in addressing the issues of rotation, scaling, shape, noise, and positioning. Validation was measured using two types of similarity measurements: pixel-difference based and human visual system (HVS) which mimics human perception in evaluating the quality of an image. The pixel-difference based metric consists of Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR) while the HVS consists of Universal Image Quality Index (UIQI) and Structural Similarity (SSIM). Our method gives the best quality of generated photos for all measures (844.04 for MSE, 19.06 for PSNR, 0.47 for UIQI, and 0.66 for SSIM).

Original languageEnglish
Pages (from-to)233-238
Number of pages6
JournalInternational Conference on Electrical Engineering, Computer Science and Informatics (EECSI)
Volume4
DOIs
Publication statusPublished - Sept 2017

Keywords

  • Colorization
  • Deep convolutional neural networks
  • Sketch inversion

Fingerprint

Dive into the research topics of 'Sketch plus colorization deep convolutional neural networks for photos generation from sketches'. Together they form a unique fingerprint.

Cite this