Abstract

Jelle Scheurleer¹, Eliana Maria Vasquez Osorio², Esther Assendelft³, Arjan Bel⁴, Irma van Dijk⁴, Harmen Bijwaard¹, Marcel van Herk⁵

¹Inholland University of Applied Sciences, Faculty of Health, Sports and Social Work, Medical Technology Research Group, Haarlem, The Netherlands; ²University of Manchester, Division of Cancer Sciences, Faculty of Biology, Medicine and Health , Manchester, United Kingdom; ³Inholland University of Applied Sciences, Faculty of Health, Sports and Social Work, Haarlem, The Netherlands; ⁴Amsterdam University Medical Centres, location AMC, Department of Radiation Oncology, Amsterdam, The Netherlands; ⁵University of Manchester, Division of Cancer Sciences, Faculty of Biology, Medicine and Health, Manchester, United Kingdom

Accurate delineation of organs at risk (OAR) is essential in modern radiotherapy to avoid under-or overestimation of the dose in the particular organs. However, this process is sensitive to intra/inter-observer variations. The aim of this project was to develop and test a web-based tool to support delineation teaching and evaluation of delineations by a large number of students using a commercial treatment planning system (TPS).

The web-based tool (based on open-source Conquest DICOM server) provides an infrastructure for contouring studies regardless the TPS used for delineation. The system keeps track of individual students generating a separate anonymized dataset per student with a template structure to enforce consistent naming. Interfaces for the teacher were created to 1) prepare individual cases, 2) export analysis results, 3) visualise contours delineated by multiple individuals and 4) set up and apply rules for automated scoring of the student’s contours to a provided Gold Standard. Multiple delineation sessions on the same cases allow for assessing the impact of teaching or guidelines. A student interface was created to allow independent data management, i.e., download of DICOM datasets to delineate, upload of delineated structures, and access to scoring with visual feedback. This interface was embedded in a learning management system (LMS) providing seamless integration of student and course information and allowing students to download, upload and evaluate cases themselves; Fig. 1.

The prototype of the tool was launched after tests in 5 different TPSs. In spring 2020 a pilot was carried out with 9 students (Fig. 2.) delineating a pelvis and a thorax set, with 9 OARs in total. Keeping track of observers was challenging as TPS tended to modify DICOM fields, especially the patient ID. The assessment of the delineation quality is time-consuming, but the visualisation is without latency. In the academic year 2020-2021, the tool was improved and implemented in the curriculum for 3^rd and 4^th year’s students. All students were able to submit, visualise and evaluate their work without the interaction of a lecturer. In the 2^nd semester of this year, the tool will be used to give each participating student individual automated feedback. 

We developed a web-based tool integrated into an LMS that handles DICOM datasets to perform delineation, and provide automated feedback on the quality of work. Visualisation, quantification and feed-back upon delineation tasks are important means for improving the skills of students and to gain insight into the effectiveness of teaching programs. This process can support continuous professional development.