Figure 7

Alignment between FAR-Pi imaging dual visible/near infrared camera (VIS/NIR) module and surgeon’s eye is assured at any distance if VIS/NIR module and eye are optically coaxial. In A-E) the VIS/NIR module (green cube) is between the eyes (not aligned with the eye), whereas in F-J) a beamsplitter in front of the augmented reality (AR) glasses makes the VIS/NIR module and eye optically coaxial. A) Experimental schematic with a resolution target positioned at distances ranging from 40 to 60 cm away from AR glasses with VIS/NIR module between the eyes. B-C) 3D rendering of resolution target (purple) in front of glasses (B) and looking through the glasses (C) with the image captured by the VIS/NIR module projected on the glasses (green). D) A camera is placed behind the glasses to visualize what the surgeon sees, this ‘surrogate-eye’ camera output is shown in purple. With the resolution target 50 cm from glasses, VIS/NIR camera (green) images do not align with ‘surrogate-eye’ images. Matching fiducial markers define a projective alignment transformation. E) Misalignment error between transformed images and ‘surrogate-eye’ images is plotted for resolution target distances ± 10 cm from calibration distance, showing increase of 2 mm in horizontal misalignment per centimeter deviation. This misalignment is visualized above the plot. At 50 cm VIS/NIR images and the ‘surrogate-eye’ images align (0 cm box above plot), but when the resolution target is moved ± 5 cm from the calibration distance, applying the 50-cm-derived transform to the VIS/NIR images no longer achieves alignment with ‘surrogate-eye’ images (± 5 cm boxes). F-H) Similar to A-C) but now the VIS/NIR module is mounted above a beamsplitter (yellow) that is placed in front of glasses. I) With the resolution target 50 cm from the glasses, the ‘surrogate-eye’ image (purple) and VIS/NIR image (green) are more closely aligned than in D). As in D) a projective alignment transform is defined at 50 cm by matching fiducial markers. J) As in E) misalignment between transformed VIS/NIR images and ‘surrogate-eye’ images is quantified and visualized. Unlike E), images at all distances remain aligned despite the 50-cm-derived transform being applied at each distance.