126. DENSE RECONSTRUCTION AND VISUAL TRACKING FOR MINIMALLY INVASIVE SURGERY
Name: Yi Luo
Grad Year: 2017
Image guidance is an important research aspect for minimal-invasive surgery where the surgical scene is captured by laparoscopic stereo camera. Although the visual feedback provides the primary surgeon real-time perception of the intra-operating scene with depth information, the field of view is narrow and it is not capable of providing extra structure information under the non-transparent tissues. Moreover, the video streamed out for assist surgeons is two-dimensional in nature which hinders the collaboration with primary surgeon in the operation. Over the past few years, much effort has been made to investigate salient feature tracking and reconstruction of point map. However, sparse tracking is unstable in this texture-less and dynamic environment with tissue deformation and moving instrument. Sparse reconstruction of the scene also fails to provide enough visual information for assist surgeons. Thus we study and implement an SLAM (simultaneous localization and mapping) system to give a dense reconstruction of surgical scene and track camera motion robustly for minimal-invasive surgery. Dense point cloud is triangulated using stereo matching which followed by total variation optimization for smoothing. Camera position is estimated frame by frame using per-pixel tracker to iteratively minimize reprojection error. M-estimator is used in minimization to cope with outliers induced by dynamic objects. As a result, the preoperative model could be augmented into visual feedback with relative position to the camera. Additionally, 3D scene could be shown to surgeons by virtual reality or used for further field of view expansion.
Industry Application Area(s)
Life Sciences/Medical Devices & Instruments | Software, Analytics