Neural Unsigned Distance Fields for Implicit Function Learning

Julian Chibane, Aymen Mir, Gerard Pons-Moll

Max Planck Institute for Informatics, Saarland Informatics Campus, Germany

NeurIPS 2020, Vancouver, Canda
Overview: Our method can represent and reconstruct complex open surfaces. Given a sparsetest point cloud of a captured room (left) it generates a detailed, completed scene (right).
this slowpoke moves
Point Cloud Completion of a complex scene using our method

Abstract

In this work we target a learnable output representation that allows continuous, high resolution outputs of arbitrary shape. Recent works represent 3D surfaces implicitly with a Neural Network, thereby breaking previous barriers in resolution, and ability to represent diverse topologies. However, neural implicit representations are limited to closed surfaces, which divide the space into inside and outside. Many real world objects such as walls of a scene scanned by a sensor, clothing, or a car with inner structures are not closed. This constitutes a significant barrier, in terms of data pre-processing (objects need to be artificially closed creating artifacts), and the ability to output open surfaces. In this work, we propose Neural Distance Fields (NDF), a neural network based model which predicts the unsigned distance field for arbitrary 3D shapes given sparse point clouds. NDF retain the good properties of recent implicit learning methods, but do not require pre-processing, and significantly broaden the class of representable shapes. NDF represent surfaces at high resolutions as prior implicit models, but do not require closed surface data, and significantly broaden the class of representable shapes in the output. Since classical methods (such as marching cubes) for mesh reconstruction and rendering assume a signed distance field, learning NDF poses additional challenges. To reconstruct a dense point-cloud, we simply sample points in space and query the distance and gradient values analytically using NDF; then we project points to the surface by shifting them along the gradient d distance units. NDF allow to extract the surface as very dense point clouds and as meshes. We also show that NDF allow for surface normal calculation and can be rendered using a slight modification of sphere tracing and can be used for multimodal regression (multiple outputs for one input) with techniques that have been exclusively used for rendering in graphics. We find NDF can be used for multi-target regression (multiple outputs for one input) with techniques that have been exclusively used for rendering in graphics. Experiments on ShapeNet show that NDF, while simple, is the state-of-the art, and allows to reconstruct shapes with inner structures, such as the chairs inside a bus. Notably, we show that NDF are not restricted to 3D shapes, and can approximate more general open surfaces such as curves, manifolds, and functions.

Comparison of the representation capacity of NDF vs Occupancy and SDF based methods. Recent works rely on occupancies or signed distances to represent surfaces, which limits shapes to be closed. In NDF, we learn with an un-signed distance field representation, allowing us to reconstruct a broader class of shapes
Overview of Point Cloud Inference, visualized on a 2D slice of a 3D bus. a) A sparse input is given. b) For each point in 3D, the unsigned distance field is predicted from the inputwith NDF. This yields a continuously completedrepresentation of arbitrary resolution andtopology. c) The corresponding gradient field of NDF can be elegantly computed analytically with back-propagation. Gradients pointing towards the depth direction appear as a dot. d) A point p in 3D space, is moved f(p) units in the negative gradient direction to yield its predicted closest surface point q
3D Shape Reconstruction of Closed Surfaces In order to be able to compare to the state of the art - Occupancy Networks, IF-Nets, PSGN DMC, we train all on 3094 ShapeNetcars pre-processed to be closed. This step looses all interior structures. We show reconstruction results when the input is 300 points, and 3000 points respectively. Point clouds are generated by sampling the closed Shapenet car models.
3D Shape Reconstruction of Complex Shapes. Reconstruction results on all classes of closed ShapeNet data from 3000 pointstrained with a single model
3D Shape Reconstruction of Complex Shapes. Point Cloud Completion of garments, with 3000 input points, visualized using dense output point cloud generation direct tracing. Unlike all priorimplicit formulations, our method can represent open surfaces.
Point cloud reconstruction of inner-structures on a test set car. Ours is the only method to successfully reconstruct full inner-structure. SAL and ours directly trained on raw data. IF-Net trained on closed data (without inner structure) for reference. SAL, like our method, does not require preprocessed water-tight data. However, unlike our method, it's final output is an SDF, which is limited to closed surfaces and can not represent the inner structures of cars.

Citation

@inproceedings{chibane2020ndf,
    title = {Neural Unsigned Distance Fields for Implicit Function Learning},
    author = {Chibane, Julian and Mir, Aymen and Pons-Moll, Gerard},
    booktitle = {Advances in Neural Information Processing Systems ({NeurIPS})},
    month = {December},
    year = {2020},
}