TY - JOUR
T1 - Sketch plus colorization deep convolutional neural networks for photos generation from sketches
AU - Putri, Vinnia Kemala
AU - Fanany, Mohamad Ivan
N1 - Publisher Copyright:
© 2018, Institute of Advanced Engineering and Science. All rights reserved.
PY - 2017/9
Y1 - 2017/9
N2 - In this paper, we introduce a method to generate photos from sketches using Deep Convolutional Neural Networks (DCNN). This research proposes a method by combining a network to invert sketches into photos (sketch inversion net) with a network to predict color given grayscale images (colorization net). By using this method, the quality of generated photos is expected to be more similar to the actual photos. We first artificially constructed uncontrolled conditions for the dataset. The dataset, which consists of hand-drawn sketches and their corresponding photos, were pre-processed using several data augmentation techniques to train the models in addressing the issues of rotation, scaling, shape, noise, and positioning. Validation was measured using two types of similarity measurements: pixel-difference based and human visual system (HVS) which mimics human perception in evaluating the quality of an image. The pixel-difference based metric consists of Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR) while the HVS consists of Universal Image Quality Index (UIQI) and Structural Similarity (SSIM). Our method gives the best quality of generated photos for all measures (844.04 for MSE, 19.06 for PSNR, 0.47 for UIQI, and 0.66 for SSIM).
AB - In this paper, we introduce a method to generate photos from sketches using Deep Convolutional Neural Networks (DCNN). This research proposes a method by combining a network to invert sketches into photos (sketch inversion net) with a network to predict color given grayscale images (colorization net). By using this method, the quality of generated photos is expected to be more similar to the actual photos. We first artificially constructed uncontrolled conditions for the dataset. The dataset, which consists of hand-drawn sketches and their corresponding photos, were pre-processed using several data augmentation techniques to train the models in addressing the issues of rotation, scaling, shape, noise, and positioning. Validation was measured using two types of similarity measurements: pixel-difference based and human visual system (HVS) which mimics human perception in evaluating the quality of an image. The pixel-difference based metric consists of Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR) while the HVS consists of Universal Image Quality Index (UIQI) and Structural Similarity (SSIM). Our method gives the best quality of generated photos for all measures (844.04 for MSE, 19.06 for PSNR, 0.47 for UIQI, and 0.66 for SSIM).
KW - Colorization
KW - Deep convolutional neural networks
KW - Sketch inversion
UR - http://www.scopus.com/inward/record.url?scp=85044851169&partnerID=8YFLogxK
U2 - 10.11591/eecsi.4.1040
DO - 10.11591/eecsi.4.1040
M3 - Article
AN - SCOPUS:85044851169
SN - 2407-439X
VL - 4
SP - 233
EP - 238
JO - International Conference on Electrical Engineering, Computer Science and Informatics (EECSI)
JF - International Conference on Electrical Engineering, Computer Science and Informatics (EECSI)
ER -