Department: Computer Science & Engineering
Research Institute Affiliation: Center for Visual Computing
Faculty Advisor(s): Ravi Ramamoorthi

Primary Student
Name: Sai Bi
Email: sabi@ucsd.edu
Phone: 858-352-8857
Grad Year: 2020

Student Collaborators
Nima Khademi Kalantari, khademi.nima@gmail.com

Modelling a real world scene is an important problem with a variety of applications including video games, virtual reality, and animations. Recently, with the advent of consumer depth cameras, 3D reconstruction of scene's geometry has become widespread. These cameras typically take a set of RGB-D images of the scene from different viewpoints. The geometry can then be reconstructed using existing methods from the depth images. However, to fully reproduce a real scene's appearance, a high-quality texture map should be produced from the input RGB photographs. This is often challenging because of several reasons including the inaccuracies of the reconstructed geometries and the estimated camera poses. Although this problem has been extensively studied in the past, existing approaches are not able to handle the cases where the reconstructed geometry and estimated poses are highly inaccurate. In these cases, these methods usually produce results with severe blurring and ghosting artifacts. To address this problem, we propose a novel patch-based optimization system which integrates the mapping into the energy formulation. We optimize the proposed energy equation with a two step approach which involves a patch search/vote and color mapping. As a result of this optimization, we synthesize an image for each input photograph in a way that all the synthesized images satisfy the photo consistency of the mapping. Our method is able to produce visually-pleasing artifact-free texture maps on poorly reconstructed geometries. Furthermore, the flexibility of our patch-based optimization system allows us to use it for other applications such as multi-view camouflage and to perform editing tasks on the texture maps (e.g., hole-filling and reshuffling).

Industry Application Area(s)
Internet, Networking, Systems | computer vision, compute graphics

« Back to Posters or Search Results

Contact:   researchexpo@soe.ucsd.edu   (858) 534-6068