TY - JOUR
T1 - Learning GANs in Simultaneous Game Using Sinkhorn with Positive Features
AU - Adnan, Risman
AU - Saputra, Muchlisin Adi
AU - Fadlil, Junaidillah
AU - Ezerman, Martianus Frederic
AU - Iqbal, Muhamad
AU - Basaruddin, Tjan
N1 - Publisher Copyright:
Author
PY - 2021
Y1 - 2021
N2 - Entropy regularized optimal transport (EOT) distance and its symmetric normalization, known as the Sinkhorn divergence, offer smooth and continuous metrized weak-convergence distance metrics. They have excellent geometric properties and are useful to compare probability distributions in some generative adversarial network (GAN) models. Computing them using the original Sinkhorn matrix scaling algorithm is still expensive. The running time is quadratic at O(n2) in the size n of the training dataset. This work investigates the problem of accelerating the GAN training when Sinkhorn divergence is used as a minimax objective. Let G be a Gaussian map from the ground space onto the positive orthant Rr + with r ≪ n. To speed up the divergence computation, we propose the use of c(x, y) = -ε log ⟨G(x), G(y)⟩ as the ground cost. This approximation, known as Sinkhorn with positive features, brings down the running time of the Sinkhorn matrix scaling algorithm to O(r n), which is linear in n. To solve the minimax optimization in GAN, we put forward a more efficient simultaneous stochastic gradient descent-ascent (SimSGDA) algorithm in place of the standard sequential gradient techniques. Empirical evidence shows that our model, trained using SimSGDA on the DCGAN neural architecture on tiny-coloured Cats and CelebA datasets, converges to stationary points. These are the local Nash equilibrium points. We carried out numerical experiments to confirm that our model is computationally stable. It generates samples of comparable quality to those produced by prior Sinkhorn and Wasserstein GANs. Further simulations, assessed on the similarity index measures (SSIM), show that our model’s empirical convergence rate is comparable to that of WGAN-GP.
AB - Entropy regularized optimal transport (EOT) distance and its symmetric normalization, known as the Sinkhorn divergence, offer smooth and continuous metrized weak-convergence distance metrics. They have excellent geometric properties and are useful to compare probability distributions in some generative adversarial network (GAN) models. Computing them using the original Sinkhorn matrix scaling algorithm is still expensive. The running time is quadratic at O(n2) in the size n of the training dataset. This work investigates the problem of accelerating the GAN training when Sinkhorn divergence is used as a minimax objective. Let G be a Gaussian map from the ground space onto the positive orthant Rr + with r ≪ n. To speed up the divergence computation, we propose the use of c(x, y) = -ε log ⟨G(x), G(y)⟩ as the ground cost. This approximation, known as Sinkhorn with positive features, brings down the running time of the Sinkhorn matrix scaling algorithm to O(r n), which is linear in n. To solve the minimax optimization in GAN, we put forward a more efficient simultaneous stochastic gradient descent-ascent (SimSGDA) algorithm in place of the standard sequential gradient techniques. Empirical evidence shows that our model, trained using SimSGDA on the DCGAN neural architecture on tiny-coloured Cats and CelebA datasets, converges to stationary points. These are the local Nash equilibrium points. We carried out numerical experiments to confirm that our model is computationally stable. It generates samples of comparable quality to those produced by prior Sinkhorn and Wasserstein GANs. Further simulations, assessed on the similarity index measures (SSIM), show that our model’s empirical convergence rate is comparable to that of WGAN-GP.
KW - Costs
KW - Entropy regularized optimal transport
KW - Games
KW - generative adversarial network
KW - Generative adversarial networks
KW - Kernel
KW - Numerical models
KW - Optimization
KW - Sinkhorn divergence
KW - Sinkhorn with positive features
KW - Training
UR - http://www.scopus.com/inward/record.url?scp=85117836478&partnerID=8YFLogxK
U2 - 10.1109/ACCESS.2021.3120128
DO - 10.1109/ACCESS.2021.3120128
M3 - Article
AN - SCOPUS:85117836478
JO - IEEE Access
JF - IEEE Access
SN - 2169-3536
ER -