TY - GEN
T1 - Sketch plus colorization deep convolutional neural networks for photos generation from sketches
AU - Putri, Vinnia Kemala
AU - Fanany, Mohamad Ivan
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2017/12/22
Y1 - 2017/12/22
N2 - In this paper, we introduce a method to generate photos from sketches using Deep Convolutional Neural Networks (DCNN). This research proposes a method by combining a network to invert sketches into photos (sketch inversion net) with a network to predict color given grayscale images (colorization net). By using this method, the quality of generated photos is expected to be more similar to the actual photos. We first artificially constructed uncontrolled conditions for the dataset. The dataset, which consists of hand-drawn sketches and their corresponding photos, were pre-processed using several data augmentation techniques to train the models in addressing the issues of rotation, scaling, shape, noise, and positioning. Validation was measured using two types of similarity measurements: pixel-difference based and human visual system (HVS) which mimics human perception in evaluating the quality of an image. The pixel-difference based metric consists of Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR) while the HVS consists of Universal Image Quality Index (UIQI) and Structural Similarity (SSIM). Our method gives the best quality of generated photos for all measures (844.04 for MSE, 19.06 for PSNR, 0.47 for UIQI, and 0.66 for SSIM).
AB - In this paper, we introduce a method to generate photos from sketches using Deep Convolutional Neural Networks (DCNN). This research proposes a method by combining a network to invert sketches into photos (sketch inversion net) with a network to predict color given grayscale images (colorization net). By using this method, the quality of generated photos is expected to be more similar to the actual photos. We first artificially constructed uncontrolled conditions for the dataset. The dataset, which consists of hand-drawn sketches and their corresponding photos, were pre-processed using several data augmentation techniques to train the models in addressing the issues of rotation, scaling, shape, noise, and positioning. Validation was measured using two types of similarity measurements: pixel-difference based and human visual system (HVS) which mimics human perception in evaluating the quality of an image. The pixel-difference based metric consists of Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR) while the HVS consists of Universal Image Quality Index (UIQI) and Structural Similarity (SSIM). Our method gives the best quality of generated photos for all measures (844.04 for MSE, 19.06 for PSNR, 0.47 for UIQI, and 0.66 for SSIM).
KW - Colorization
KW - Deep convolutional neural networks
KW - Sketch inversion
UR - http://www.scopus.com/inward/record.url?scp=85046408246&partnerID=8YFLogxK
U2 - 10.1109/EECSI.2017.8239116
DO - 10.1109/EECSI.2017.8239116
M3 - Conference contribution
AN - SCOPUS:85046408246
T3 - International Conference on Electrical Engineering, Computer Science and Informatics (EECSI)
BT - Proceedings - 2017 4th International Conference on Electrical Engineering, Computer Science and Informatics, EECSI 2017
A2 - Rahmawan, Hatib
A2 - Facta, Mochammad
A2 - Riyadi, Munawar A.
A2 - Stiawan, Deris
PB - Institute of Advanced Engineering and Science
T2 - 4th International Conference on Electrical Engineering, Computer Science and Informatics, EECSI 2017
Y2 - 19 September 2017 through 21 September 2017
ER -