模块化神经图像信号处理

摘要

本文提出一种模块化神经图像信号处理（ISP）框架，能够处理原始传感器数据并生成高质量显示参考图像。与现有神经ISP设计方案不同，本方法引入高度模块化架构，实现对渲染流程中多个中间阶段的完整控制。该模块化设计不仅实现了高精度渲染，还显著提升了系统的可扩展性、可调试性、对未见过相机型号的泛化能力以及匹配不同用户偏好风格的灵活性。为验证该设计的优势，我们开发了基于本神经ISP的用户交互式照片编辑工具，支持多样化编辑操作与图片风格。该工具经过精心设计，既能充分发挥神经ISP的高质量渲染特性，又可实现无限次的后编辑重渲染。本方法采用全学习型框架，提供不同计算容量的模型变体（完整流程参数量约0.5M至3.9M），在多个测试集上均能稳定呈现具有竞争力的定性与定量结果。补充视频请参阅：https://youtu.be/ByhQjQSjxVM

English

This paper presents a modular neural image signal processing (ISP) framework that processes raw inputs and renders high-quality display-referred images. Unlike prior neural ISP designs, our method introduces a high degree of modularity, providing full control over multiple intermediate stages of the rendering process.~This modular design not only achieves high rendering accuracy but also improves scalability, debuggability, generalization to unseen cameras, and flexibility to match different user-preference styles. To demonstrate the advantages of this design, we built a user-interactive photo-editing tool that leverages our neural ISP to support diverse editing operations and picture styles. The tool is carefully engineered to take advantage of the high-quality rendering of our neural ISP and to enable unlimited post-editable re-rendering. Our method is a fully learning-based framework with variants of different capacities, all of moderate size (ranging from ~0.5 M to ~3.9 M parameters for the entire pipeline), and consistently delivers competitive qualitative and quantitative results across multiple test sets. Watch the supplemental video at: https://youtu.be/ByhQjQSjxVM

模块化神经图像信号处理

Modular Neural Image Signal Processing

摘要

Support