This project for CS684 – Introduction to Computer Vision was focused on 3D reconstruction of geometry from many images. Below (left), we can see one of the images provided of a corner of a temple. The first step was to use a segmentation algorithm on the images (below, right).

Then, each of the segmented images was provided to a visual hull algorithm. Imagine that the purpose of this algorithm is to carve a sculpture out of a block of marble. In our algorithm, we take the block and subdivide it into many voxels. Then it “looks” at the reference model from each of the provided perspectives (each of the provided photos) and if the segmentation algorithm marked a section as empty, then the algorithm marks the corresponding voxels as empty.

There was also a photoconsistency aspect to this algorithm. Once we have the visual hull of the object, it is possible to examine the surface voxels and to determine whether they should be discarded or kept. This is achieved by looking at the position of that voxel in each of the reference images that can see it. The algorithm essentially checks if the colour of the surface is the same in each of the images. If not, it is likely that the surface is not at that point and the voxel is discarded. That’s the basic idea anyways.

Below is the reconstructed geometry, rendered using voxels. It would be possible to run marching cubes to get a mesh, but that wasn’t necessary for this project.

Below is my project report. It provides slightly more details about the project and the code can be found there.