Session Item

Multiple tools and metrics are available for validation (commissioning) and verification (quality assurance) of deformable image registration (DIR), all with their own advantages and disadvantages in the context of radiotherapy. The use of tools should depend on the available input, desired output, time requirement, application and/or anatomical area. We present a consensus on reasoning for and recommendations on the choice of tools for validation and verification of DIR in specific situations.

Discussions were hosted by the ESTRO Physics Workshop in 2021 on Commissioning and Quality Assurance for DIR in Radiotherapy. We made a comprehensive overview of available metrics and tools and defined inputs, outputs, and limitations. A consensus was then reached on what (complementary) requirements were needed for either validation or verification of DIR for applications like contour-propagation or dose accumulation, and what tools and metrics are associated with this.

An overview of the input, output, timing and general objective for a selection of metrics is given in Table 1. As all considered metrics have disadvantages and certain insensitivities, a combination using different inputs and/or assumptions should be used. Evaluations should often include alignment metrics (e.g. image similarity, contour-overlap, target registration error) and deformation vector field (DVF) consistency/plausibility metrics (e.g. inverse consistency, Jacobian determinant).

For validation, we recommend the target registration error of a set of manually annotated anatomical landmarks and the distance-to-agreement of manually delineated contours. These should be supplemented by plausibility metric(s) to reduce their disadvantage of being sensitive only locally in high-contrast regions. Digital phantoms are useful for validation (especially of DIR for dose accumulation) but are currently not available for a wide range of anatomies, image modalities and types of deformations.

For patient-specific verification of DIR for contour propagation, we recommend at least visual inspection of the registered image and contour. For patient-specific verification of DIR for dose warping and accumulation, we recommend at least visual inspection of the DVF and/or an image similarity map, together with the Jacobian determinant and the distance-to-dose-difference or dose (gradient) maps. These recommendations also hold when warping quantitative information like Hounsfield units or PET data. General objectives for these metrics are also given in Table 1. We acknowledge that some of these metrics are still missing in some commercial softwares.

Table 1. Overview of some metrics and their characteristics

We provide reasoning and recommendations for using a set of tools and metrics for specific situations of validation and/or verification of DIR for radiotherapy and acknowledge the need for a broader incorporation of these metrics in commercial software and for more complex and specific (digital) phantoms.