ESTRO 2023

Session Item

Head and neck

Session Type: Poster (Digital)

Track: Clinical

Journey:

Systematic review of the methodological conduct of Head and Neck SCC outcome prediction models

, Denmark

Presentation Number: PO-1236

Abstract

Abstract Title:

Systematic review of the methodological conduct of Head and Neck SCC outcome prediction models

Authors:

Farhannah Aly^1,2,3, Christian Rønn Hansen^4,5,6,7, Daniel Al Mouiee^1,8,9, Purnima Sundaresan^10,11, Ali Haidar^1,2, Shalini Vinod^2,12, Lois Holloway^13,14,3,7

¹Ingham Institute for Applied Medical Research, Medical Physics Research Group, Liverpool, Australia; ²University of New South Wales, Southwest Sydney Clinical Campus, Sydney, Australia; ³Liverpool and Macarthur Cancer Therapy Centres, Department of Radiation Oncology, Liverpool, Australia; ⁴Odense University Hospital, Laboratory of Radiation Physics, Odense, Denmark; ⁵University of Southern Denmark, Department of Clinical Research, Odense, Denmark; ⁶Aarhus University Hospital, Danish Centre for Particle Therapy, Aarhus , Denmark; ⁷Institute of Medical Physics, University of Sydney, School of Physics, Sydney, Australia; ⁸University of New South Wales , Southwest Sydney Clinical Campus, Sydney, Australia; ⁹Liverpool and Macarthur Cancer Therapy Centres , Department of Radiation Oncology, Liverpool, Australia; ¹⁰Western Sydney Local Health District, Sydney West Radiation Oncology Network, Sydney, Australia; ¹¹The University of Sydney, Sydney Medical School, Sydney, Australia; ¹²Liverpool and Macarthur Cancer Therapy Centres, Department of Radiation Oncology , Liverpool, Australia; ¹³Ingham Institute for Applied Medical Research, Medical Physics Research Group , Liverpool, Australia; ¹⁴University of New South Wales, Southwest Sydney Clinical Campus , Sydney, Australia

Show Affiliations

Purpose or Objective

Multiple outcome prediction models have been developed for Head and Neck Squamous Cell Carcinoma (HNSCC). This systematic review aimed to identify HNSCC outcome prediction model studies that can be used at time of treatment-decision making, and assess their methodological quality, thus making recommendations for clinical practice.

Material and Methods

Inclusion criteria were mucosal HNSCC prognostic prediction model studies (development and/or validation), published between 2000-2022, that incorporated clinically available variables and predicted tumour-related outcomes. Eligible publications were identified from PubMed and Embase. Methodological quality and risk of bias were assessed using the checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) and prediction model risk of bias assessment tool (PROBAST). Eligible publications were categorised by study type for reporting.

Results

64 eligible publications were identified, 55 publications reported model developments and 37 reported external validations. 28 publications reported both model development and external validation. Publication characteristics are shown in table 1. CHARMS checklist items relating to participants, predictors, outcomes, handling of missing data, and some model development and evaluation procedures were generally well-reported. Less well-reported were measures accounting for model overfitting and model performance measures, especially model calibration. Full model information was poorly reported (three of 55 model developments), specifically model intercept, baseline survival or full model code. Most publications (54 of 55 model developments, 28 of 37 model external validations) were found to be at high risk of bias, predominantly due to methodological issues in the PROBAST participants and analysis domains (Figure 1). Factors contributing to high risk of bias included low sample size, inappropriate categorisation of continuous predictors, excluding participants in the analysis, inappropriate handling of participants with missing data, selecting predictors based on univariable analysis and not appropriately accounting for overfitting, underfitting and optimism in model performance.

Conclusion

Most eligible outcome prediction models contained methodological issues, introducing a high risk of bias, which may affect accuracy in heterogeneous populations. Inadequate reporting of complete model information was also demonstrated. Careful critical appraisal of outcome prediction model publications should be undertaken before clinical use. It is also recommended that future prediction model studies use the TRIPOD reporting guidelines to ensure that sufficient information is provided to allow for critical appraisal and independent external validation. Independent external validation studies in the local population and demonstration of clinical impact are essential for the clinical implementation of outcome prediction models.