Convergence of Non-Convex Non-Concave GANs Using Sinkhorn Divergence

Risman Adnan, Muchlisin Adi Saputra, Junaidillah Fadlil, Martianus Frederic Ezerman, Muhamad Iqbal, Tjan Basaruddin

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

Sinkhorn divergence is a symmetric normalization of entropic regularized optimal transport. It is a smooth and continuous metrized weak-convergence with excellent geometric properties. We use it as an alternative for the minimax objective function in formulating generative adversarial networks. The optimization is defined with Sinkhorn divergence as the objective, under the non-convex and non-concave condition. This work focuses on the optimization's convergence and stability. We propose a first order sequential stochastic gradient descent ascent (SeqSGDA) algorithm. Under some mild approximations, the learning converges to local minimax points. Using the structural similarity index measure (SSIM), we supply a non-asymptotic analysis of the algorithm's convergence rate. Empirical evidences show a convergence rate, which is inversely proportional to the number of iterations, when tested on tiny colour datasets Cats and CelebA on the deep convolutional generative adversarial networks and ResNet neural architectures. The entropy regularization parameter $\varepsilon $ is approximated to the SSIM tolerance $\epsilon $. We determine that the iteration complexity to return to an $\epsilon $ -stationary point to be $\mathcal {O}\left ({\kappa \, \log (\epsilon ^{-1})}\right)$ , where $\kappa $ is a value that depends on the Sinkhorn divergence's convexity and the minimax step ratio in the SeqSGDA algorithm.

Original languageEnglish
Article number9410544
Pages (from-to)67595-67609
Number of pages15
JournalIEEE Access
Volume9
DOIs
Publication statusPublished - 2021

Keywords

  • Convergence
  • generative adversarial networks
  • optimal transport
  • Sinkhorn divergence

Fingerprint

Dive into the research topics of 'Convergence of Non-Convex Non-Concave GANs Using Sinkhorn Divergence'. Together they form a unique fingerprint.

Cite this