Fig. 1

Flowchart of the stitching pipeline. Modules are depicted in gray, with data transfers indicated by yellow parallelograms. RGB frames are represented in green, and HSI in blue. Note that HSI lines are depicted with an illustrative width for clarity, although they are 1 pixel wide in reality. Timestamps are omitted for simplification. (a) This module receives and preprocesses two asynchronous raw data streams from the laparoscope and forwards them separately. (b) The picking module extracts salient keypoints from the current RGB frame \(I_i\) and matches them with its predecessor \(I_{i-1}\). Sanity checks associated with this frame-to-frame registration enable the selection of the best keypoints. (c) The coordinates and descriptors of the most robust keypoints from frame \(I_i\) are transferred to the global registration module, which maintains a database of accumulated keypoints since sequence start. A brute force matcher establishes correspondence between current and stored keypoints, enabling computation of a global frame-to-panorama homography \({H}_{(i, P)}\). (d) The homography is adjusted for application on HSI lines and used to warp and stitch the buffered set of corresponding lines \(\tilde{\mathcal {L}}_{i}\) onto the panorama. (e) The visualization module renders the HSI panorama as an overlay on the RGB video. This figure has been designed using an icon from Flaticon.com