幂等生成网络

摘要

我们提出了一种基于训练神经网络成为幂等的生成建模新方法。幂等算子是指可以连续应用而不改变结果超出初始应用的算子，即f(f(z))=f(z)。所提出的模型f被训练来将源分布（例如，高斯噪声）映射到目标分布（例如，逼真图像），使用以下目标：(1) 目标分布中的实例应映射到它们自身，即f(x)=x。我们将目标流形定义为所有f映射到自身的实例的集合。(2) 形成源分布的实例应映射到定义的目标流形上。这是通过优化幂等性项f(f(z))=f(z)来实现的，这鼓励f(z)的范围在目标流形上。在理想假设下，这样的过程可以被证明收敛到目标分布。这种策略导致了一个能够在一步中生成输出的模型，保持一致的潜在空间，同时也允许进行顺序应用以进行细化。此外，我们发现通过处理来自目标和源分布的输入，该模型能够熟练地将损坏或修改的数据投影回目标流形。这项工作是通向“全局投影器”的第一步，它使得能够将任何输入投影到目标数据分布中。

English

We propose a new approach for generative modeling based on training a neural network to be idempotent. An idempotent operator is one that can be applied sequentially without changing the result beyond the initial application, namely f(f(z))=f(z). The proposed model f is trained to map a source distribution (e.g, Gaussian noise) to a target distribution (e.g. realistic images) using the following objectives: (1) Instances from the target distribution should map to themselves, namely f(x)=x. We define the target manifold as the set of all instances that f maps to themselves. (2) Instances that form the source distribution should map onto the defined target manifold. This is achieved by optimizing the idempotence term, f(f(z))=f(z) which encourages the range of f(z) to be on the target manifold. Under ideal assumptions such a process provably converges to the target distribution. This strategy results in a model capable of generating an output in one step, maintaining a consistent latent space, while also allowing sequential applications for refinement. Additionally, we find that by processing inputs from both target and source distributions, the model adeptly projects corrupted or modified data back to the target manifold. This work is a first step towards a ``global projector'' that enables projecting any input into a target data distribution.

幂等生成网络

Idempotent Generative Network

摘要

Support