文章信息

标题

Extracting Triangular 3D Models, Materials, and Lighting From Images

从图像中提取三维三角面片模型，材质，亮度

作者

Douglas E. Zongker 1 Dawn M. Werner 1 Brian Curless1 David H. Salesin 1,2

1 University of Washington 2 Microsoft Research

发表信息

Jacob Munkberg 1 Jon Hasselgren 1 Tianchang Shen 1,2,3 Jun Gao 1,2,3 Wenzheng Chen 1,2,3 Alex Evans 1 Thomas M ̈ uller 1 Sanja Fidler 1,2,3

1 NVIDIA 2 University of Toronto 3 Vector Institute

基本上是英伟达的全员参与。Vector Institute是多伦多大学创建的向量学院。

本文发表收录于2022年的CVPR。

引用信息

截至2023年12月2日，被引用次数为154次。

@InProceedings{Munkberg_2022_CVPR,
    author    = {Munkberg, Jacob and Hasselgren, Jon and Shen, Tianchang and Gao, Jun and Chen, Wenzheng and Evans, Alex and M\"uller, Thomas and Fidler, Sanja},
    title     = {Extracting Triangular 3D Models, Materials, and Lighting From Images},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {8280-8290}
}

论文链接

后人对此文章的评价

文章内容

摘要

We present an efficient method for joint optimization of topology, materials and lighting from multi-view image observations. Unlike recent multi-view reconstruction approaches, which typically produce entangled 3D representations encoded in neural networks, we output triangle meshes with spatially-varying materials and environment lighting that can be deployed in any traditional graphics engine unmodified. We leverage recent work in differentiable rendering, coordinate-based networks to compactly represent volumetric texturing, alongside differentiable marching tetrahedrons to enable gradient-based optimization directly on the surface mesh. Finally, we introduce a differentiable formulation of the split sum approximation of environment lighting to efficiently recover all-frequency lighting. Experiments show our extracted models used in advanced scene editing, material decomposition, and high quality view interpolation, all running at interactive rates in triangle-based renderers (rasterizers and path tracers).

我们提出了一个有效从多视角图像中联合优化拓扑、材质以及光照的方法。不溶于最近的多视角重建方法，这些方法经常生成耦合起来的三维表征，表征通常编码于神经网络中。我们的输出是三角面片以及其随着空间变化的材质、环境光，可以直接用于传统的计算机图形学引擎中，不用做修改调整。我们采用最近可微分渲染的工作，基于坐标的网络以精简地表达体积纹理，沿着可微分的四面体移动方法，保证基于梯度的优化，该优化直接作用于物体表面的三角面片。最后，我们引入了一个可微分的公式，使用分总的方法近似环境光，以高效地恢复所有频率的光照。实验表明我们提取到的模型可以用于先进的场景编辑，材质解耦合，以及高质量的视角插值。所有的训练都在可以交互的速率上，使用基于三角的渲染器（光栅器和光路追踪器）。

介绍

英伟达做的工作。

基于BRDF、符号距离函数做了设计和优化，同时优化面片表面的几何结构以及光度场的估计情况。

056 Extracting Triangular 3D Models, Materials, and Lighting From Images