real-time photo-realistic augmented reality under dynamic environment lighting

Department: Computer Science & Engineering
Research Institute Affiliation: Center for Visual Computing
Faculty Advisor(s): Jurgen P. Schulze | Ravi Ramamoorthi

Primary Student
Name: Kamran Alipour
Phone: 858-361-7853
Grad Year: 2022

Recent developments in graphics hardware has made realistic rendering a possibility in real-time applications. One of the most applicable techniques in photo-realistic augmented reality (AR) is to capture the environments lighting and reflect that lighting on synthetic objects in a way that augmented objects look as part of the real scene coherently. We have developed a realtime photo-realistic AR module using an analytic expression for the irradiance in terms of spherical harmonic (SH) coefficients of the lighting. By computing only 9 coefficients based on low frequencies in the lighting, SH technique provides an efficient rendering algorithm for diffuse objects to be utilized for realtime applications. We have used a Ricoh Theta S 360 camera to acquire live dynamic lighting from the surrounding environment. The algorithm is successfully implemented within C++ platform in integration with OpenCV and OpenGL to provide an AR experience for an Oculus Rift headset. The module is also successfully tested within Unity 3D platform along with Vuforia package to produce AR applications for Microsoft Hololens.

« Back to Posters or Search Results