Session Item

Co-registration between in vivo radiological imaging and ex vivo histopathologic Whole Slide Image (WSI) enables pixel-wise mapping of ROIs. It brings biological insights for radiation oncologists towards new guidelines and homogenization of clinical practice across centers. In addition, AI-based segmentation methods often fail to learn ROIs on radiology due to interobserver contour variability and can benefit from these cross-modality ground truth labels. However, in addition to the differences in resolution scales, color intensities, and nature of data (2D vs. 3D), the task is often manual and challenging because the tissue undergoes severe deformation and shrinkage during the histological process. We propose a cutting-edge deep-learning framework for the automatic registration of 2D histopathology with 3D radiology. The pipeline is generalizable to any radiological modality and allows the reconstruction of synthetic 3D histology.

We collected a cohort from 77 patients (joint collaboration Gustave Roussy Institute - Centre Léon Bérard, France) on whom were acquired both pre-operative H&N 3D CT scans and 4 to 11 digitalized 2D WSIs after total laryngectomy. Our novelty is two-fold: first, to solve the multimodal issue, we developed a transfer model based on cycleGAN to bring both images to the same modality. Second, for the 2D-3D problem, we built a slice-to-volume unsupervised registration pipeline. Both blocks are trained in parallel for mutual benefit (Figure 1). Moreover, correct mapping is difficult to achieve without proper initialization because of the tissue artifacts mentioned above. To overcome it, we guided the registration by rigidly aligning both thyroid and cricoid cartilages which are supposedly not distorted during tissue preparation procedures.

Figure 2 highlights some visual samples. The deformed WSI (c) is closely aligned to the fixed CT scan (b), for rigid regions like cartilage as well as for soft tissue at the edge of slide inclusion. Quantitatively for all patients, we report a mean Dice Score of 0.91/1 between cartilage masks and a mean Normalized Cross Correlation of 0.89/1 across all tissue. For the modality transfer generation, we can reconstruct synthetic images with a Structure Similarity index of 0.82/1, enabling a 3D reconstruction of the histology specimen by filling empty slices with generated histology from CT.

Co-registration between 3D radiology and 2D histology is a challenging problem, especially for H&N for which the larynx on WSI is often halved. We solve it through an automatic deep learning framework, which is by construction also able of generating synthetic images from one modality to the other one. It paves the way for a better understanding of the tissue microenvironment by overlaying histology-based features on the radiological acquisition, while the reconstruction task enables the generation of noninvasive virtual biopsy. Such tools should yield more precise and personalized RT treatment.