基于退化模型的多路径扩散可调谐超透镜摄影技术

摘要

超透镜在超紧凑计算成像领域展现出巨大潜力，但面临着复杂光学退化和计算复原难度的挑战。现有方法通常依赖于精确的光学校准或大规模配对数据集，这对实际成像系统而言并非易事。此外，缺乏对推理过程的控制往往导致不理想的幻觉伪影。我们提出了基于退化建模的多路径扩散方法，用于可调谐超透镜摄影，利用预训练模型中的强大自然图像先验，而非依赖大规模数据集。我们的框架采用正向、中性和负向提示路径，以平衡高频细节生成、结构保真度以及超透镜特有退化的抑制，同时结合伪数据增强技术。可调谐解码器实现了保真度与感知质量之间的可控权衡。此外，空间变化退化感知注意力（SVDA）模块自适应地建模复杂的光学和传感器引起的退化。最后，我们设计并构建了毫米级MetaCamera进行实际验证。大量实验结果表明，我们的方法超越了现有最先进技术，实现了高保真度和锐利的图像重建。更多资料请访问：https://dmdiff.github.io/。

English

Metalenses offer significant potential for ultra-compact computational imaging but face challenges from complex optical degradation and computational restoration difficulties. Existing methods typically rely on precise optical calibration or massive paired datasets, which are non-trivial for real-world imaging systems. Furthermore, a lack of control over the inference process often results in undesirable hallucinated artifacts. We introduce Degradation-Modeled Multipath Diffusion for tunable metalens photography, leveraging powerful natural image priors from pretrained models instead of large datasets. Our framework uses positive, neutral, and negative-prompt paths to balance high-frequency detail generation, structural fidelity, and suppression of metalens-specific degradation, alongside pseudo data augmentation. A tunable decoder enables controlled trade-offs between fidelity and perceptual quality. Additionally, a spatially varying degradation-aware attention (SVDA) module adaptively models complex optical and sensor-induced degradation. Finally, we design and build a millimeter-scale MetaCamera for real-world validation. Extensive results show that our approach outperforms state-of-the-art methods, achieving high-fidelity and sharp image reconstruction. More materials: https://dmdiff.github.io/.

基于退化模型的多路径扩散可调谐超透镜摄影技术

Degradation-Modeled Multipath Diffusion for Tunable Metalens Photography

摘要

Support