K-文化的一切——从回归到 K-美妆，发送到您的邮箱订阅邮件

METAL MEDIA

TT-net: Quantum Inspired Tensor Network Denoising in Conditional GANs

arXiv:2608.197892026-08-21

让图像各通道互相比对信息,GAN去噪效果更好

这篇论文提出了TT-Net,用张量列车分解(一种源自量子多体物理模拟的技术)替换了以往条件GAN去噪模型中使用的SVD(奇异值分解,一种将矩阵拆解压缩的线性代数工具)滤波模块。此前的SVD-Net对特征图的每个通道单独做分解,通道之间完全无法比较信息,而TT-Net通过两步分解让不同通道的信息可以相互对照。在CIFAR-10图像上分别加入高斯噪声、运动模糊和椒盐噪声进行测试,TT-Net在全部三种噪声下的PSNR和SSIM指标均优于SVD-Net。

METAL MEDIA 解读图

让图像各通道互相比对信息,GAN去噪效果更好

  1. 01SVD-Net对特征图的每个通道单独做SVD分解,无法利用通道间的关联;TT-Net改用两步张量列车分解,让通道信息可以直接互相比较
  2. 02实验采用受控对比:生成器、判别器和训练设置完全一致,唯一变化的是去噪模块从SVD换成张量列车分解
  3. 03相比SVD-Net,TT-Net在高斯噪声上PSNR提升2.68dB、SSIM提升0.0513,运动模糊上PSNR提升8.32dB、SSIM提升0.3798,椒盐噪声上PSNR提升0.99dB、SSIM提升0.0165
  4. 04在高斯噪声测试中,TT-Net的表现也超过了作为参照对比的EigenGAN以及当前较先进的Pix2pix方法
  5. 05TT-Net的对抗损失项在三种噪声下都很早就趋于停滞,但图像重建质量仍持续提升,这留下了一个尚未解决的问题:对抗训练部分究竟发挥了多大作用
这是 METAL MEDIA 制作的解读图，并非论文作者提供的原图。

他们做了什么

  1. SVD-Net对特征图的每个通道单独做SVD分解,无法利用通道间的关联;TT-Net改用两步张量列车分解,让通道信息可以直接互相比较
  2. 实验采用受控对比:生成器、判别器和训练设置完全一致,唯一变化的是去噪模块从SVD换成张量列车分解
  3. 相比SVD-Net,TT-Net在高斯噪声上PSNR提升2.68dB、SSIM提升0.0513,运动模糊上PSNR提升8.32dB、SSIM提升0.3798,椒盐噪声上PSNR提升0.99dB、SSIM提升0.0165
  4. 在高斯噪声测试中,TT-Net的表现也超过了作为参照对比的EigenGAN以及当前较先进的Pix2pix方法
  5. TT-Net的对抗损失项在三种噪声下都很早就趋于停滞,但图像重建质量仍持续提升,这留下了一个尚未解决的问题:对抗训练部分究竟发挥了多大作用
Figure 1: Representative test-set outputs across all four models and three noise types. Ground truth is shown once per row since it is identical across models. Each model’s own noisy input and output are shown alongside it.
Figure 1: Representative test-set outputs across all four models and three noise types. Ground truth is shown once per row since it is identical across models. Each model’s own noisy input and output are shown alongside it.
Figure 2: Discriminator output probabilities D⁡(real)/D⁡(fake) over training, SVD-Net vs. TT-Net, by noise type.
Figure 2: Discriminator output probabilities D⁡(real)/D⁡(fake) over training, SVD-Net vs. TT-Net, by noise type.
Table 1: Feature map shape at each TT-Net / SVD-Net insertion point.
Insertion pointChannelsHeightWidth
After 1st encoder stage643232
After 2nd encoder stage1281616
After 3rd encoder stage25688
Figure 3: Total generator loss (ℒG=ℒGa​d​v+ℒrecon, Section 4.2) over training, SVD-Net (top) vs. TT-Net (bottom), by noise type, linear scale.
Figure 3: Total generator loss (ℒG=ℒGa​d​v+ℒrecon, Section 4.2) over training, SVD-Net (top) vs. TT-Net (bottom), by noise type, linear scale.
Figure 4: Generator loss components (Ga​d​v, L2, MS-SSIM, recon) over training, SVD-Net (top) vs. TT-Net (bottom), by noise type. See Section 4.2 for the loss definitions, ℒrecon=δ⋅ℒMS-SSIM+(1−δ)⋅ℒL​2,δ=0.84.
Figure 4: Generator loss components (Ga​d​v, L2, MS-SSIM, recon) over training, SVD-Net (top) vs. TT-Net (bottom), by noise type. See Section 4.2 for the loss definitions, ℒrecon=δ⋅ℒMS-SSIM+(1−δ)⋅ℒL​2,δ=0.84.
Table 2: Optimized hyperparameters used for each model, with a uniform batch size of 32 applied across all four.
Modellr (G, D)β1β2Optimizer
Pix2pix2×10−40.50.999Adam
EigenGAN2×10−30.50.999Adam
SVD-Net1×10−40.90.999Adam
TT-Net1×10−40.90.999Adam
Figure 5: Validation PSNR over training, SVD-Net vs. TT-Net, by noise type.
Figure 5: Validation PSNR over training, SVD-Net vs. TT-Net, by noise type.
Figure 6: Validation SSIM (windowed) over training, SVD-Net vs. TT-Net, by noise type.
Figure 6: Validation SSIM (windowed) over training, SVD-Net vs. TT-Net, by noise type.
Table 3: Test-set PSNR and SSIM (standard windowed formulation) across noise types, using each model’s optimized hyperparameters (Section 4.1) with a uniform batch size of 32. Bold indicates the best value across all four models. TT-Net achieves the best PSNR and SSIM of any model on Gaussian noise. †SVD-Net’s motion-blur result reflects a late-training discriminator collapse rather than a converged model (Section 5.3).
GaussianMotion BlurSalt & Pepper
ModelPSNRSSIMPSNRSSIMPSNRSSIM
Pix2pix25.520.746636.180.964341.790.9881
EigenGAN25.180.754526.710.839834.240.9513
SVD-Net23.600.745518.02†0.4260†28.980.8761
TT-Net26.280.796826.340.805829.970.8926
Figure 7: MS-SSIM loss component over training, SVD-Net vs. TT-Net, by noise type.
Figure 7: MS-SSIM loss component over training, SVD-Net vs. TT-Net, by noise type.

为什么重要

这项工作说明,只需针对性地改进去噪机制、让模型能够比较不同通道的信息,就能在不改变GAN整体框架的情况下明显提升图像修复质量。它也展示了原本用于模拟量子系统的张量网络工具,可以作为实用的特征滤波器应用到真实的深度学习任务中。

本文术语

  • 张量列车(Tensor Train) · 将多维数组(张量)拆解为一系列小矩阵乘积的方法,在量子物理中也称为矩阵乘积态
  • SVD(奇异值分解) · 将一个矩阵拆分成三个较简单矩阵乘积的线性代数方法,常用于保留矩阵中最重要的信息
  • GAN(生成对抗网络) · 由生成图像的生成器和判断真假的判别器相互对抗训练组成的模型
  • PSNR/SSIM · 衡量修复图像与原始图像相似程度的常用指标,数值越高表示效果越好
  • 判别器崩溃 · 训练过程中判别器压制生成器,导致生成结果质量突然大幅下降的现象

论文原文摘要（英文）

Developed as a workhorse for classical simulations of quantum algorithms and quantum many-body systems, Tensor Network methods have entered the scientific mainstream in quantum physics. Among various types of tensor networks, Tensor Trains (commonly know as Matrix Product States in the quantum computing community) have already found applications in machine learning. These methods often rely on a powerful linear algebra tool called the Singular Value Decomposition (SVD). Several conditional GAN architectures for image denoising incorporate SVD as a single-cut decomposition step applied to generator feature maps. In this work we introduce TT-Net, which replaces the per-channel SVD denoising block with a two-cut tensor-train decomposition capable of accessing cross-channel information directly, a capability absent from contemporary alternatives. In a controlled comparison differing only in this decomposition mechanism, TT-Net outperforms SVD-Net on PSNR and SSIM across all three noise types tested (Gaussian, motion blur, and salt-and-pepper), supporting the hypothesis that cross-channel access improves denoising quality. Training-dynamics analysis further shows that TT-Net's adversarial loss term consistently saturates to a stagnant state across all three noise types, more so than SVD-Net's, while reconstruction quality continues to improve regardless, raising an open question about the adversarial component's contribution that this work identifies but does not resolve. Furthermore, for Gaussian noise our method outperforms both the EigenGAN and the state of the art Pix2pix method which does not assume any linear algebra decompositions and does not retain any linear algebra information. Our manuscript shows how quantum inspired tools can be used as practical real world feature filters for deep learning applications.

作者 · Michal A. Sterzel, Marko J. Ran\v{c}i\'c

在 arXiv 阅读

最新论文

全部论文 →

METAL MEDIA 最新报道

图片来源： Michal A. Sterzel et al., arXiv:2608.19789, CC BY 4.0