real-time photo-realistic augmented reality under dynamic environment lighting
Name: Kamran Alipour
Grad Year: 2022
Recent developments in graphics hardware has made realistic rendering a possibility in real-time applications. One of the most applicable techniques in photo-realistic augmented reality (AR) is to capture the environments lighting and reflect that lighting on synthetic objects in a way that augmented objects look as part of the real scene coherently. We have developed a realtime photo-realistic AR module using an analytic expression for the irradiance in terms of spherical harmonic (SH) coefficients of the lighting. By computing only 9 coefficients based on low frequencies in the lighting, SH technique provides an efficient rendering algorithm for diffuse objects to be utilized for realtime applications. We have used a Ricoh Theta S 360 camera to acquire live dynamic lighting from the surrounding environment. The algorithm is successfully implemented within C++ platform in integration with OpenCV and OpenGL to provide an AR experience for an Oculus Rift headset. The module is also successfully tested within Unity 3D platform along with Vuforia package to produce AR applications for Microsoft Hololens.