3D Shape reconstruction from a single image. This project integrates parametric surfaces (radial basis functions) with optimization and illumination estimation. The goal is to rebuild 3D surfaces from a single image with minimum user input (brush strokes).

Real time capture of my implentation. On the left we can see a single image as input. On the right, the 3D reconstruction.

This project, automatic 3D shape reconstruction from a single image, is my version of Parametric Shape-from-Shading by Radial Basis Functions (Wei et al.) combined with our light detection algorithm. 

Input image and evolution of the parametric surface generated by our method. Real time capturefrom video footage.

One of the key advantages of this method is that it can use additional input in order to improve the results. The system allows for depth, normal, contour and equality constraints. This optimization scheme, by design, is specially suitable for sparse constraint input (such as user strokes) and the system will incorporate this data into the solver, refining the surface in a smooth and continuous fashion.

Top Row: Left: new 3D views of the surface generated from the constraints and the David image. You can observe how the scale of the depth is corrected from the surface shown in the video. Right: Novel views of surfaces generated from the sombrero image. The rightmost image was generated including the equality constraints defined in the bottom row. Bottom row: Left: Input image (low resolution fragment of David statue) and user strokes dening depth constraints (dark-is-deep). Right: Input image (low resolution hat) and user strokes denoting similar-height pixels (equality constraint).

One of the experiments ahead is to combine this method with sparse or incomplete 3D depth information like the one provided by devices such as Microsoft Kinect. 

Left: Example of depthmap obtained from the Kinect camera.Middle, Right: Additional examples,shown as point clouds, Note the sparsity of the data, very suitable for RBF interpolation. Images from KyleMcDonald. Used by permission (CC-BY-SA-NC).