模块化神经图像信号处理

摘要

本文提出了一种模块化神经图像信号处理（ISP）框架，能够处理原始传感器数据并生成高质量显示参考图像。与现有神经ISP设计不同，我们的方法通过高度模块化实现了对成像流程中多个中间阶段的完全可控。这种模块化设计不仅实现了高精度渲染，还显著提升了系统的可扩展性、可调试性、对未见过相机型号的泛化能力以及匹配不同用户偏好风格的灵活性。为验证该设计优势，我们开发了基于此神经ISP的用户交互式照片编辑工具，支持多样化编辑操作与图片风格调整。该工具经过精心设计，既能充分发挥神经ISP的高质量渲染特性，又可实现无限次的后置可编辑重渲染。我们的方法采用全学习型框架，提供不同计算规模的模型变体（完整流程参数量约0.5M至3.9M），在多个测试集上均能稳定输出具有竞争力的定性与定量结果。补充视频请参见：https://youtu.be/ByhQjQSjxVM

English

This paper presents a modular neural image signal processing (ISP) framework that processes raw inputs and renders high-quality display-referred images. Unlike prior neural ISP designs, our method introduces a high degree of modularity, providing full control over multiple intermediate stages of the rendering process.~This modular design not only achieves high rendering accuracy but also improves scalability, debuggability, generalization to unseen cameras, and flexibility to match different user-preference styles. To demonstrate the advantages of this design, we built a user-interactive photo-editing tool that leverages our neural ISP to support diverse editing operations and picture styles. The tool is carefully engineered to take advantage of the high-quality rendering of our neural ISP and to enable unlimited post-editable re-rendering. Our method is a fully learning-based framework with variants of different capacities, all of moderate size (ranging from ~0.5 M to ~3.9 M parameters for the entire pipeline), and consistently delivers competitive qualitative and quantitative results across multiple test sets. Watch the supplemental video at: https://youtu.be/ByhQjQSjxVM

模块化神经图像信号处理

Modular Neural Image Signal Processing

摘要

Support