Overview of SNGAN (Spectral Normalization GAN), algorithms and implementation examples

Machine Learning Artificial Intelligence Digital Transformation Natural Language Processing Image Processing Reinforcement Learning Probabilistic Generative Modeling Deep Learning Python Navigation of this blog
Overview of SNGAN (Spectral Normalization GAN)

SNGAN (Spectral Normalization GAN) is a method that introduces spectral normalization to stabilize the training of GAN (Generative Adversarial Network) as described in “Overview of GANs and Various Applications and Implementations”. This approach aims to suppress gradient explosion and disappearance and stabilize learning by applying spectral normalization to the discriminator weight matrix in particular.

In training GANs, the Lipschitz constraints (smoothness of the function) of the discriminator (D) are important. In particular, in Wasserstein GAN (WGAN), weight clipping and gradient penalty (Gradient Penalty) were used to satisfy the Lipschitz constraint. However, these methods had the following problems

  • Weight clipping: Overconstrainting reduces expressive power and makes it difficult to maintain appropriate Lipschitz constraints.
  • Gradient Penalty (WGAN-GP): requires additional computation in the cost function and slows down learning.

SNGAN solves these problems by constraining the maximum singular value of the discriminator’s weight matrix to be less than 1, a method that satisfies the Lipschitz constraint.

SNGAN applies the following normalization to the weight matrix \(W\) of each fully coupled or convolutional layer of the discriminator \[\hat{W}=\frac{W}{\sigma(W)}\]

where \(\sigma(W)\) is the maximum singular value (spectral norm) of the weight matrix\(W\).

The maximum singular value is obtained using the following eigenvalue decomposition. \[\sigma(W)=\max_{||v|||_2=1}||W_U||||_2\] This normalization limits the Lipschitz constant for each layer, preventing the discriminator from becoming too sensitive and leading to gradient stabilization.

Advantages of SNGAN include.

(1) Stable learning

  • Mode collapse is easily mitigated because the Lipschitz constraints can be properly maintained.
  • Fast learning because no additional calculation of gradient penalty is required as in WGAN-GP.

(2) Easy to handle

  • No additional hyper-parameter adjustment (clipping range and penalty factor) is required.
  • Easily applicable to existing GAN architectures.

(3) High quality image generation

  • It has been confirmed that SNGAN can generate sharper images on image datasets such as CIFAR-10 and ImageNet.

SNGAN implementations use convolutional and all-combining layers with spectral normalization applied to the discriminators of ordinary GANs.

SNGAN is a powerful method for stabilizing GAN training and has become a method incorporated in many modern GAN architectures.

Implementation Example

An example implementation of a Spectral Normalization GAN (SNGAN) is shown below. Spectral normalization is applied to the Discriminator, and the Generator has the structure of a standard GAN. 1.

1. Installation of necessary libraries: First, install the necessary libraries.

pip install torch torchvision matplotlib

2. SNGAN Implementation: The following code implements and trains the SNGAN Generator and Discriminator.

import torch
import torch.nn as nn
import torch.optim as optim
import torchvision
import torchvision.transforms as transforms
import matplotlib.pyplot as plt

# ===========================
#  1. Hyper-parameter setting
# ===========================
latent_dim = 100  # Dimensions of latent variables
image_size = 64   # Image size (64x64)
batch_size = 128
num_epochs = 20
lr = 0.0002
beta1 = 0.5  # Hyperparameters for Adam optimizer

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# ===========================
#  2. Data Set Preparation (CIFAR-10)
# ===========================
transform = transforms.Compose([
    transforms.Resize(image_size),
    transforms.ToTensor(),
    transforms.Normalize([0.5], [0.5])  # Normalized to [-1, 1
])

dataset = torchvision.datasets.CIFAR10(root="./data", download=True, transform=transform)
dataloader = torch.utils.data.DataLoader(dataset, batch_size=batch_size, shuffle=True)

# ===========================
#  Definition of Generator
# ===========================
class Generator(nn.Module):
    def __init__(self):
        super(Generator, self).__init__()
        self.model = nn.Sequential(
            nn.ConvTranspose2d(latent_dim, 512, 4, 1, 0, bias=False),
            nn.BatchNorm2d(512),
            nn.ReLU(True),

            nn.ConvTranspose2d(512, 256, 4, 2, 1, bias=False),
            nn.BatchNorm2d(256),
            nn.ReLU(True),

            nn.ConvTranspose2d(256, 128, 4, 2, 1, bias=False),
            nn.BatchNorm2d(128),
            nn.ReLU(True),

            nn.ConvTranspose2d(128, 64, 4, 2, 1, bias=False),
            nn.BatchNorm2d(64),
            nn.ReLU(True),

            nn.ConvTranspose2d(64, 3, 4, 2, 1, bias=False),
            nn.Tanh()  # 出力を [-1, 1] にする
        )

    def forward(self, z):
        return self.model(z)

# ===========================
#  4. Discriminator Definition (Spectral Normalization)
# ===========================
class Discriminator(nn.Module):
    def __init__(self):
        super(Discriminator, self).__init__()
        self.model = nn.Sequential(
            nn.utils.spectral_norm(nn.Conv2d(3, 64, 4, 2, 1, bias=False)),
            nn.LeakyReLU(0.2, inplace=True),

            nn.utils.spectral_norm(nn.Conv2d(64, 128, 4, 2, 1, bias=False)),
            nn.BatchNorm2d(128),
            nn.LeakyReLU(0.2, inplace=True),

            nn.utils.spectral_norm(nn.Conv2d(128, 256, 4, 2, 1, bias=False)),
            nn.BatchNorm2d(256),
            nn.LeakyReLU(0.2, inplace=True),

            nn.utils.spectral_norm(nn.Conv2d(256, 512, 4, 2, 1, bias=False)),
            nn.BatchNorm2d(512),
            nn.LeakyReLU(0.2, inplace=True),

            nn.Conv2d(512, 1, 4, 1, 0, bias=False)
        )

    def forward(self, x):
        return self.model(x).view(-1, 1).squeeze(1)  # (batch, 1) → (batch,)

# ===========================
#  5. Model and Optimizer Settings
# ===========================
netG = Generator().to(device)
netD = Discriminator().to(device)

criterion = nn.BCEWithLogitsLoss()  # BCE loss + logit output

optimizerG = optim.Adam(netG.parameters(), lr=lr, betas=(beta1, 0.999))
optimizerD = optim.Adam(netD.parameters(), lr=lr, betas=(beta1, 0.999))

# ===========================
#  6. learning loop
# ===========================
fixed_noise = torch.randn(16, latent_dim, 1, 1, device=device)

for epoch in range(num_epochs):
    for i, (real_images, _) in enumerate(dataloader):
        real_images = real_images.to(device)
        batch_size = real_images.size(0)

        # Label Creation
        real_labels = torch.ones(batch_size, device=device)
        fake_labels = torch.zeros(batch_size, device=device)

        # === Learning discriminator (D) ===
        optimizerD.zero_grad()
        outputs = netD(real_images)
        loss_real = criterion(outputs, real_labels)
        loss_real.backward()

        noise = torch.randn(batch_size, latent_dim, 1, 1, device=device)
        fake_images = netG(noise)
        outputs = netD(fake_images.detach())  # Stop G gradient
        loss_fake = criterion(outputs, fake_labels)
        loss_fake.backward()

        optimizerD.step()

        # === Generator (G) Learning ===
        optimizerG.zero_grad()
        outputs = netD(fake_images)
        loss_G = criterion(outputs, real_labels)
        loss_G.backward()
        optimizerG.step()

        # Output in progress
        if i % 500 == 0:
            print(f"Epoch [{epoch+1}/{num_epochs}], Step [{i}/{len(dataloader)}], "
                  f"D Loss: {loss_real.item() + loss_fake.item():.4f}, G Loss: {loss_G.item():.4f}")

    # Image Generation and Display
    with torch.no_grad():
        fake_images = netG(fixed_noise).cpu()
    grid = torchvision.utils.make_grid(fake_images, normalize=True)
    plt.figure(figsize=(6,6))
    plt.imshow(grid.permute(1, 2, 0))
    plt.title(f"Epoch {epoch+1}")
    plt.axis("off")
    plt.show()

Key points of implementation

  • Apply nn.utils.spectral_norm() to the discriminator (Discriminator) to introduce spectral normalization.
  • Generator uses the usual convolutional transposition layer (ConvTranspose2d).
  • BCEWithLogitsLoss() is used for the loss function, and the output of the discriminator is processed as is without using a sigmoid function.
  • The Adam optimizer is used to stabilize the training.

After executing this code, stable training with SNGAN allows for high-quality image generation using the CIFAR-10 dataset.

Application Examples

SNGAN is a type of GAN in which learning is stabilized by introducing spectral normalization and mode decay is suppressed. SNGAN is capable of high-quality image generation and is used in the following areas.

1. image generation (generation of high-resolution images)

Case study: Anime face image generation (Danbooru dataset)

Abstract: SNGAN has been applied to face generation for anime characters; training on the Danbooru dataset enables high-quality anime character faces to be generated.

Technologies Used

    • Dataset: Danbooru (animated face images)
    • Model: SNGAN
    • Applications: VTuber avatar generation, character design support

Reference: In the paper by Miyato et al. (2018), experiments on the generation of anime face images using SNGAN are conducted.

2. medical image generation/complementation

Case study: noise removal and generation of MRI and CT images

Abstract: SNGAN is also used to remove noise and complement missing parts of medical images; when data from MRI and CT scans are missing, SNGAN can be used to generate realistic images and complement the data.

Technologies Used

    • Dataset: Brain MRI, Chest X-ray
    • Model: SNGAN (high-quality generation through discriminator stabilization)
    • Applications: MRI super-resolution (low resolution)
      • MRI super-resolution (conversion of low-resolution scan images to high-resolution)
      • Disease simulation (generate simulated images of abnormal areas)
      • Data enhancement (application in areas where medical datasets are scarce)

Reference: study by A. Mahapatra et al. (2019) on super-resolution of medical images using SNGAN.

3. fashion design

Case study: clothing design generation (FashionGAN)

Abstract: SNGAN is also used to generate new clothing designs and fashion styles.
In particular, SNGAN is used in the field of virtual fashion design using GAN, where it is important to stabilize the discriminator.

Technologies used

    • Dataset: DeepFashion, Zalando
    • Model: SNGAN
    • Applications
      • Generation of new fashion designs (brand ideation)
      • Virtual try-on system (to generate designs tailored to customer preferences)
      • Clothing image enhancement for e-commerce sites

Reference: FashionGAN research by Liu et al. (2016)

4. generation of AI art

Case study: Generation of abstract paintings and works of art

Abstract: SNGAN has been applied to generate realistic paintings and abstract art because of its stable discriminator. For example, it is possible to generate unique works of art using SNGAN trained in Picasso-style and Hokusai-style.

Technologies Used

    • Dataset: Museum painting data (e.g. WikiArt)
    • Model: SNGAN (stable learning with enhanced discriminators)
    • Applications
      • Creation of new artworks by AI
      • NFT art generation
      • Learning a specific painter’s style and creating new artworks

Reference: Also applied in DeepArt, Runway ML, and other GAN-based art generation platforms

5. 3D object generation

Case study: 3D model generation in games and metaverse

Abstract: SNGAN is not only used to generate 2D images, but also 3D objects. For example, SNGAN is effective in face generation of 3D characters using GANs, because stable learning is required.

Technologies Used

    • Dataset: 3D Face Dataset, ShapeNet
    • Model: SNGAN (combined with 3D GAN)
    • Applications
      • Generation of metaverse avatars
      • Creation of realistic game character faces
      • Automatic generation of 3D objects (buildings, furniture, etc.)

Reference: 3D GAN research by Wu et al. (2016)

reference book

References related to SNGAN (Spectral Normalization GAN) are listed below. 1.

1. the basics of SNGAN and related papers

Papers

Takeru Miyato et al, “Spectral Normalization for Generative Adversarial Networks” (ICLR 2018)
[URL]: https://arxiv.org/abs/1802.05957
Abstract: We propose a method that uses spectral normalization to strengthen the Lipschitz constraint of discriminators and enable stable learning. Experiments demonstrated high-quality image generation on CIFAR-10, STL-10, ImageNet, etc.

2. fundamentals of GAN

Book: 

Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play

Author: David Foster
Publisher: O’Reilly Media (2019)
Summary: A broad introduction to GANs, covering not only SNGANs but also other methods such as StyleGANs and CycleGANs. Plenty of examples of implementations with code.

Practical Deep Learning for Computer Vision with Python

Deep Learning with Python, Second Edition

Author: François Chollet (Keras developer)
Publisher: Manning Publications (2021)
Description: Explains the basic concepts of GANs and how to implement them using TensorFlow / Keras.

3. applications and implementation of GANs

Hands-On Image Generation with TensorFlow: A practical guide to GANs, VAEs, and Diffusion Models

Author: Soon Yau Cheong
Publisher: Packt Publishing (2023)
Description: Covers various GAN methods for image generation, with code implementations using PyTorch and TensorFlow; describes implementation techniques (regularization and stabilization methods) related to SNGANs.

GANs in Action: Deep Learning with Generative Adversarial Networks

Authors: Jakub Langr, Vladimir Bok
Publisher: Manning Publications (2019)
Abstract: A broad overview of GANs, from basic mechanics to applications, with chapters on “Discriminator Normalization” and “Stabilization Methods,” which are the foundation of SNGANs.

An Introduction to Deep Reinforcement Learning with TensorFlow.

4. learn about research and state-of-the-art methods

Advances in Deep Learning for Medical Image Analysis

Authors: Archana Mire, Shadma Anwer, Pradeep Singh
Publisher: Academic Press (2022)
Abstract: Introduces the use of GANs in medical image generation, mentioning stable generation methods using regularization of discriminators such as SNGANs.

Machine Learning with PyTorch and Scikit-Learn

Authors: Sebastian Raschka, Yuxi (Hayden) Liu
Publisher: Packt Publishing (2022)
Description: Provides a detailed introduction to the theory of GANs and their implementation in PyTorch, as well as useful techniques for implementing SNGANs (spectral normalization, learning stabilization).

    コメント

    Exit mobile version
    タイトルとURLをコピーしました