Laparoscopic surgery is a Minimally Invasive Surgical (MIS) procedure using endoscopes with small incisions to carry out internal operations on patients. A video image from a camera at the end of the endoscope provide the surgeon with visual feedback from inside the patient. While MIS offers considerable advantages over open surgeries, it also imposes challenges on surgeons' performance due to issues associated with the field of view (FOV), hand-eye disalignment and disorientation. Augmented Reality (AR) technology can help overcome the limitations by overlaying additional information with the real scene through augmentation of target surgical locations, annotations, labels, tumour measurement or even 3D reconstruction of anatomic structures.
With our colleagues at Bournemouth University we are working to provide AR in MIS for both monocular and stereoscopic endoscopes. Our framework provides real-time computation of 3D markerless tracking through a robust feature-based simultaneous localization and mapping (SLAM) approach. An interactive geometry-aware AR environment is implemented by incrementally building a geometric mesh via zero mean normalised cross correlation (ZNCC) stereo matching.
Monocular MIS surgical scenes are particularly challenging to augment with correct spatial calibration. We are also addressing this problem so as to prevent drifting and inaccurate planar mapping.
Collaborators: Bournemouth University