Automatic 3D reconstruction and rendering based on sparse images

Abstract

3D object reconstruction and rendering have a vital role in computer vision due to its wide range of applications. There are several techniques to achieve this, including laser scanner and photogrammetry. In recent years, artificial intelligence-based methods, such as neural radiance field, have been proposed and received impressive results. This paper presents a method that applies photogrammetry for automatic 3D point reconstruction of an object, and followed by rendering using Gaussian splatting. In our work, we first capture images of an object using normal cameras. The capture path involves moving 360 degrees around the object at 3 different heights. Second, structure from the motion-based method is employed to reconstruct its 3D point cloud on the surface of the object. Next, the 3D point cloud is converted to and considered as initial 3D Gaussian points. Then, these Gaussian points are trained based on the difference between projection syntheses of Gaussian points on training images. Finally, a rendering image for a novel view is produced by projection syntheses of Gaussian splatting on that surface. Our experiments showed that this flow-chart gives superior results on 3D reconstruction and rendering. With a small object, specially a  fabric flower model with the size 50cm x 40cm x 40cm, it is necessary to use approximately 90 training images, and the processing time needed to generate the 3D Gaussian point cloud for rendering is about 30 minutes.